NAVAL 

POSTGRADUATE 

SCHOOL 

MONTEREY,  CALIFORNIA 

THESIS 


AIRCRAFT  PILOT  SITUATIONAL  AWARENESS 
INTERFACE  FOR  AIRBORNE  OPERATION  OF 
NETWORK  CONTROLLED  UNMANNED  SYSTEMS  (US) 

by 

Ibrahim  Demirel 
March  2008 

Thesis  Advisor:  Alex  Bordetsky 

Co-Advisor:  Eugene  Bourakov 


Approved  for  public  release;  distribution  is  unlimited 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


REPORT  DOCUMENTATION  PAGE 


Form  Approved  OMB  No.  0704-0188 

Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instruction, 
searching  existing  data  sources,  gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send 
comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including  suggestions  for  reducing  this  burden,  to 
Washington  headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA 
22202-4302,  and  to  the  Office  of  Management  and  Budget,  Paperwork  Reduction  Project  (0704-0188)  Washington  DC  20503. 

I.  AGENCY  USE  ONLY  (Leave  blank)  2.  REPORT  DATE  3.  REPORT  TYPE  AND  DATES  COVERED 

March  2008  Master’s  Thesis 

4.  TITLE  AND  SUBTITLE  Aircraft  Pilot  Situational  Awareness  Interface  for  5.  FUNDING  NUMBERS 

Airborne  Operation  of  Network  Controlled  Unmanned  Systems  (US) _ 

6.  AUTHOR(S)  Ibrahim  Demirel  _ 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES)  8.  PERFORMING  ORGANIZATION 

Naval  Postgraduate  School  REPORT  NUMBER 

9.  SPONSORING  /MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES)  10.  SPONSORING/MONITORING 
N/A  AGENCY  REPORT  NUMBER 

II.  SUPPLEMENTARY  NOTES  The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not  reflect  the  official  policy 

or  position  of  the  Department  of  Defense  or  the  U.S.  Government, _ 

12a.  DISTRIBUTION  /  AVAILABILITY  STATEMENT  12b.  DISTRIBUTION  CODE 

Approved  for  public  release;  distribution  is  unlimited 

13.  ABSTRACT  (maximum  200  words) 

This  thesis  research  is  focused  on  Network  Centric  Operations  with  Unmanned  Systems  (US).  It  specifically 
focuses  on  the  currently  underdeveloped  area  of  aircraft  pilot  decision  support  for  operating  USs,  including 
Unmanned  Aerial  Vehicles  (UAV),  Unmanned  Ground  Vehicles  (UGV)  and  Unmanned  Surface  Vehicles  (USV), 
over  the  network  from  the  board  of  an  aircraft. 

Building  on  Landreth  and  Glass’s  thesis  on  controlling  UAV  over  the  network,  including  from  another 
manned  aircraft,  this  thesis  aims  to  ease  implementation  and  usage  of  the,  SA  interface.  The  SA  interface  enables  the 
operator  to  be  aware  of  what  is  going  on  around  the  Unmanned  System  while  it  is  being  operated  from  a  remote 
location,  and  to  react  in  the  best  possible  way  within  a  reasonable  amount  of  time.  The  Rascal  UAV  interface  was 
reviewed,  SA-related  problems  were  identified,  and  solutions  to  those  problems  were  proposed.  After  our  studies  we 
proposed  eight  possible  solutions  to  implement,  and  one  of  them  is  implemented  and  used.  However,  due  to  some 
problems,  we  could  not  test  all  our  solutions. _ 

14.  SUBJECT  TERMS  UAS,  UAV,  Network  Control,  Tactical  UAS/UAV,  Beyond  Line  of  Site,  15.  NUMBER  OF 

ISR,  SA,  Interface  PAGES 

117 

16.  PRICE  CODE 

17.  SECURITY  18.  SECURITY  19.  SECURITY  20.  LIMITATION  OF 

CLASSIFICATION  OF  CLASSIFICATION  OF  THIS  CLASSIFICATION  OF  ABSTRACT 

REPORT  PAGE  ABSTRACT 

Unclassified  Unclassified  Unclassified  UU 

NSN  7540-0 1  -280-5500  Standard  Form  298  (Rev.  2-89) 

Prescribed  by  ANSI  Std.  239-18 


1 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


11 


Approved  for  public  release;  distribution  is  unlimited 


AIRCRAFT  PILOT  SITUATIONAL  AWARENESS  INTERFACE  FOR 
AIRBORNE  OPERATION  OF  NETWORK  CONTROLLED  UNMANNED 

SYSTEMS  (US) 

Ibrahim  Demirel 
1st  Lieutenant,  Turkish  Air  Force 
B.S.,  Industrial  Engineering,  Turkish  Air  Force  Academy,  1999 

Submitted  in  partial  fulfillment  of  the 
requirements  for  the  degree  of 


MASTER  OF  SCIENCE  IN  INFORMATION  TECHNOLOGY  MANAGEMENT 


from  the 


NAVAL  POSTGRADUATE  SCHOOL 
March  2008 


Author:  Ibrahim  Demirel 


Approved  by:  Dr.  Alex  Bordetsky 

Thesis  Advisor 


Mr.  Eugene  Bourakov 
Co-Advisor 


Dr.  Dan  C.  Boger 

Chairman,  Department  of  Information  Sciences 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


IV 


ABSTRACT 


This  thesis  research  is  focused  on  Network  Centric  Operations  with  Unmanned 
Systems  (US).  It  specifically  focuses  on  the  currently  underdeveloped  area  of  aircraft 
pilot  decision  support  for  operating  USs,  including  Unmanned  Aerial  Vehicles  (UAV), 
Unmanned  Ground  Vehicles  (UGV)  and  Unmanned  Surface  Vehicles  (USV),  over  the 
network  from  the  board  of  an  aircraft. 

Building  on  Landreth  and  Glass’s  thesis  on  controlling  UAV  over  the  network, 
including  from  another  manned  aircraft,  this  thesis  aims  to  ease  implementation  and 
usage  of  the,  SA  interface.  The  SA  interface  enables  the  operator  to  be  aware  of  what  is 
going  on  around  the  Unmanned  System  while  it  is  being  operated  from  a  remote  location, 
and  to  react  in  the  best  possible  way  within  a  reasonable  amount  of  time.  The  Rascal 
UAV  interface  was  reviewed,  SA-related  problems  were  identified,  and  solutions  to  those 
problems  were  proposed.  After  our  studies,  we  proposed  eight  possible  solutions  to 
implement,  and  one  of  them  is  implemented  and  used.  However,  due  to  some  problems, 
we  could  not  test  all  our  solutions. 
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I.  INTRODUCTION 


Chapter  I  will  lay  the  groundwork  for  this  thesis,  including  an  introduction  to 
Situational  Awareness  (SA),  the  SA  interface  and  SA  views,  which  are  a  product  of  SA 
and  the  interface.  The  chapter  begins  with  a  discussion  of  an  NPS  thesis  written  by  Major 
Kent  A.  Landreth  and  Major  John  C.  Glass.  Landreth  and  Glass’s  thesis  developed  the 
concept  of  controlling  Unmanned  Aerial  Vehicle  (UAV)  over  the  network;  this  thesis 
extends  their  work  by  adding  a  manned  aircraft  to  the  network.  The  chapter  continues 
with  some  definitions  of  SA  from  the  related  literature,  an  analysis  of  the  useful  features 
of  those  definitions  for  the  purposes  of  this  thesis,  and  a  similar  analysis  of  the  definition 
of  interface.  The  chapter  concludes  with  a  basic  introduction  to  SA  views. 

A.  PRIOR  THESIS  DONE  ON  CONTROLLING  AN  UAV  FROM  AN 

AIRCRAFT 

A  thesis  done  by  Major  Kent  A.  Landreth  and  Major  John  C.  Glass,  entitled 
“Extending  the  Tactical  Horizon:  Networking  Aircraft  to  Enable  Persistent  Surveillance 
and  Target  Development  for  SOF,”  provided  the  idea  behind  this  thesis.  Landreth  and 
Glass  sought  a  solution  to  the  Intelligence-Surveillance-Reconnaissance  (ISR)  needs  of 
Special  Forces  while  avoiding  Task  Priority  problems  related  to  ISR  capabilities,  and 
answering  those  ISR  needs  within  the  Special  Forces  unit  command.  They  stated  that, 
“Our  goal  is  to  increase  the  ISR  capacity  of  units  who  normally  would  not  rate  the 
priority  to  task  a  Predator,  Global  Hawk,  or  U-2.  There  are  two  guiding  tenets  in 
developing  this  concept.  First,  the  equipment  and  its  control  should  be  organic  to  the 
Special  Operation  Forces  (SOF)  unit  or  task  force.  Second,  utilizing  this  capability  will 
not  require  the  soldier  to  carry  any  additional  equipment  into  the  field.” 

Landreth  and  Glass  point  out  that  a  “a  gap  exists  between  what  is  needed  by 
special  operations  personnel  to  develop  potential  targets,  and  what  is  available  to  them”1 


1  Kent  A.  Landreth  and  John  C.  Glass,  “Extending  the  Tactical  Horizon:  Networking  Aircraft  to 
Enable  Persistent  Surveillance  and  Target  Development  for  SOF,”  (Master’s  thesis,  Naval  Postgraduate 
School,  2006),  xiii. 


1 


They  explain  the  reason  for  this  gap:  “Virtually  every  asset  which  can  provide  FMV  or 
even  near-real-time  imagery  over  the  horizon  or  Beyond-Line-of-Sight  (BLOS)  falls  into 
the  category  of  Low  Density  High  Demand  (LD/HD)  systems”  and  “the  demand  for  the 
ISR  products  of  these  assets  (U-2,  Global  Hawk,  Predator,  etc.)  outweighs  the  limited 
capacity  of  the  relatively  low  number  of  these  very  expensive  platforms.”2 

They  developed  a  candidate  solution.  “We  propose  in  this  paper  to  introduce  a 
concept  by  which  a  SOF  task  force  or  smaller  organization  can  utilize  existing,  low-cost 
technologies  with  Line-of-Sight  (LOS)  links  to  obtain  a  BLOS  surveillance  capability  for 
the  purpose  of  developing  potential  targets  or  objectives,  without  tasking  theater-level  or 
above  assets.”3  This  proposed  solution  requires  that  a  network  of  UAV(s)  and  manned 
aircraft  be  established  to  link  LOS  assets  in  order  to  create  a  BLOS  capability. 
Furthermore  the  real  gain  and  the  usability  comes  from  the  fact  that  “this  capability  can 
be  organic  to  the  task  force  or  lower  headquarters,  thereby  increasing  that  unit’s 
autonomy  in  launching  and  tasking  its  own  aircraft,  as  well  as  exploiting  the  resultant 
Image  Intelligence  (IMINT)  or  Signal  Intelligence  (SIGINT).”4  This  solution  fills  in  the 
gap  by  employing  the  assets  in  hand,  in  a  new  and  more  effective  way. 

Landreth  and  Glass  developed  a  basic  scenario  to  test  their  idea,  which  involved  a 
series  of  control  transfers  (of  the  UAV)  between  a  ground  station,  a  manned  aircraft,  and 
a  forward  deployed  ground  team.  According  to  this  scenario,  “[t]he  UAS  will  be 
launched  manually  and  control  passed  to  the  Ground  Control  Station  (GCS),  who  will 
then  assume  aircraft/payload  (camera)  control  via  the  NPS  developed  Situational 
Awareness  software  and  server.  The  Pelican  will  assume  UAS  control  via  SA  and  recon 
the  objective  area — simulating  BLOS  from  the  GCS — and  then  send  the  UAS  to  a  pre¬ 
planned  waypoint,  where  the  ground  team  will  assume  control  through  the  SA  server  and 
return  the  UAS  to  the  objective  area  for  further  target  development.  The  Tactical 
Operations  Center  (TOC)  will  monitor  video  and  provide  scenario  updates  based  on 

2  Landreth  and  Glass,  “Extending  the  Tactical  Horizon:  Networking  Aircraft  to  Enable  Persistent 
Surveillance  and  Target  Development  for  SOF,”  3. 

3  Ibid. 

4  Ibid. 
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information  gathered  through  mesh  video  feeds.  And  the  Unmanned  Aerial  Systems 
(UAS)  control  will  then  be  returned  to  the  Pelican  and  GCS  respectively  for  a  manual 
landing  at  the  airfield.”5 6 


Figure  1.  Tactical  scenario  of  Landreth  and  Glass6 


5  Landreth  and  Glass,  “Extending  the  Tactical  Horizon:  Networking  Aircraft  to  Enable  Persistent 
Surveillance  and  Target  Development  for  SOF,”  19. 

6  Ibid. 
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Figure  2.  Tactical  scenario  in  a  more  general  form7 


7  Vladimir  N.  Dobrokhodov,  et  al.,  “New  Generation  of  Rapid  Flight  Test  Prototyping  System  for 
Small  Unmanned  Air  Vehicles,”  paper  presented  in  AIAA  Modeling  and  Simulation  Technologies 
Conference  and  Exhibit,  August  20-23,  2007,  Hilton  Head,  South  Carolina. 
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Figure  3.  Rascal  UAV  with  comm,  antennas8 


8  Landreth  and  Glass,  “Extending  the  Tactical  Horizon:  Networking  Aircraft  to  Enable  Persistent 
Surveillance  and  Target  Development  for  SOF.” 
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Pelican-Computer 
and  Operator  Station 


Figure  4.  Aircraft  control  station9 


9  Landreth  and  Glass,  “Extending  the  Tactical  Horizon:  Networking  Aircraft  to  Enable  Persistent 
Surveillance  and  Target  Development  for  SOF.” 
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Figure  5.  Assets  of  the  experiment10 

In  compliance  with  their  scenario,  Landreth  and  Glass  developed  some  objectives 
and  measures  of  performance  (MOP)  /  measures  of  evaluation  (MOE).  These  objectives 
and  (MOP)  /  (MOE)  are  as  follows: 

B.  OBJECTIVES 

•  Establish  Effective  UAS  Control  via  Mesh  network  (airframe  and  camera) 
from  Airborne  and  Remote  Ground  Stations:  Also,  add  UAS  waypoints 
and  change  flight  parameters  as  well  as  manipulate  onboard  camera  for 
target  reconnaissance. 

•  Evaluate  Video  Quality  to  Determine  Aircraft  and  Sensor  Capabilities 

•  Use  the  Mesh  Network  to  Supply  Real-Time  Video  to  Pelican,  Ground 
Team  and  TOC  for  Simulated  ISR  Target  Development 

•  Manipulate  Mesh  Nodes  to  Simulate  BLOS  between  Parties 


10  Dobrokhodov,  et  at,  “New  Generation  of  Rapid  Flight  Test  Prototyping  System  for  Small 
Unmanned  Air  Vehicles.” 
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c. 


MEASURES  OF  PERFORMANCE/  MEASURES  OF  EVALUATION 


•  Safely  Transfer  UAS  and  Camera  Control  Between  GCS,  Pelican  and 
Remote  Ground  Team 

•  Evaluate  Image  Quality  for  Target  Development  and  UAS/Sensor  Control 

•  Functionality  of  Hardware  and  Software  for  Future  Development  and 
Field  Application1 1 

In  their  self-assessment,  Fandreth  and  Glass  suggested  that  they  had  met  the 
objectives  either  completely  or  largely.  They  experienced  difficulty  in  meeting  Objective 
Three  due  to  problems  of  camera  stability;  but  they  did  meet  that  objective  in  theory). 

Fandreth  and  Glass’s  thesis  demonstrated  some  important  facts: 

•  There  is  real  potential  for  this  concept  to  yield  to  an  effective  means  by 
which  special  operators  can  significantly  increase  their  ISR  capability 
through  a  low-cost,  organic  system  of  networked  UAS  platforms. 

•  Such  a  network  requires  no  UAS  pilot  training  for  SOF  personnel,  just 
familiarity  with  whatever  mechanism  eventually  controls  the  sensor. 

•  For  the  most  part,  the  technology  exists  today  and  would  not  add  weight  to 
the  soldier’s  rucksack. 

D.  DEFINITION  OF  SA 

Any  decision  requires  a  good  information  flow  to  the  decision  maker.  With  the 
information  from  different  sources,  the  decision  maker  creates  a  mental  picture  in  his 
head,  and  the  quality  and  quantity  of  the  information  to  some  point  has  an  effect  on  the 
completeness  and  correctness  of  the  mental  picture.  (However,  it  should  be  remembered, 
that  good  information  flow  and  great  SA  do  not  guarantee  good  decisions,  since  sound 
decision-making  is  still  needed  from  the  decision  maker).  The  mental  picture  created  can 
be  taken  as  the  simple  definition  for  situational  (or  situation)  awareness.  Different 
resources  off  different  definitions  for  various  aspects  of  situational  awareness  (SA). 
Controlling  a  system  while  being  aware  of  the  environment  plays  a  major  role  in  different 
areas  and  professions,  and  research  regarding  SA  has  been  done  with  fighter  pilots,  chess 
experts,  and  nuclear  power  plant  operators,  Different  occupations  and  areas  of  interest 

11  Landreth  and  Glass,  “Extending  the  Tactical  Horizon:  Networking  Aircraft  to  Enable  Persistent 
Surveillance  and  Target  Development  for  SOF,”  45. 
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may  have  unique  definitions  for  the  given  circumstances,  however  most  of  the  elements 
will  be  the  same  due  to  the  characteristics  of  SA.  This  section  of  the  thesis  looks  at  some 
of  these  definitions,  and  points  to  important  aspects  that  will  be  useful  in  defining  SA  for 
the  pilot  of  a  UAV. 

Dominguez’s  definition  points  to  a  mental  picture  of  the  present,  built  upon  the 
past  and  used  in  the  future  perception  of  the  environment,  much  like  integrating  blocks  of 
the  past  and  the  present  for  a  complete  understanding:  “Continuous  extraction  of 
environmental  information,  integration  of  this  information  with  previous  knowledge  to 
form  a  coherent  mental  picture,  and  the  use  of  that  picture  in  directing  further  perception 
and  anticipating  future  events.”12 

In  the  case  of  Flach’s  definition,  the  dynamic  nature  of  SA  is  emphasized:  “a 
measure  of  the  degree  of  dynamic  coupling  between  a  user  and  a  particular  situation.”13 

Artman  and  Garbis  tie  dynamism  to  the  systems  making  decision,  and  define  SA 
as  an  intermediate  state  on  the  way  to  the  decision:  “Situation  Awareness  (SA)  is  an 
intennediate  state  in  the  decision-making  process  of  dynamic  systems  where  one  should 
be  able  to  comprehend  the  situation  in  order  to  make  an  appropriate  decision  for  future 
development.”14 

One  of  the  most  widely  accepted  and  quoted  definitions  belong  to  Mica  R. 
Endsley.  “Situation  awareness  is  the  perception  of  elements  in  the  environment  within  a 
volume  of  time  and  space,  the  comprehension  of  their  meaning,  and  the  projection  of 
their  status  in  the  near  future.”15 


12  Mica  R.  Endsley  and  Daniel  J.  Garland,  Situation  Awareness  Analysis  and  Measurement,  (New 
York:  Lawrence  Erlbaum  Associates,  2000),  227. 

13  “Operator  situational  awareness  and  system  safety,” 
http://ieeexplore.ieee.org/Xplore/login.isp?url=/iel5/6803/18256/00842707.pdf?arnumber=842707 

(accessed  January  15,  2008). 

14  Henrik  Artman  and  Christer  Garbis,  “Situation  Awareness  as  Distributed  Cognition,” 
http://www.nada.kth.se/~artman/Articles/Conference/ECCE98wGarbis.pdf  (accessed  January  24,  2008). 

15  M.  R.  Endsley,  “Toward  a  theory  of  situation  awareness  in  dynamic  systems,”  Human  Factors, 
37(1),  32-64,  (1995b). 


9 


In  his  definition,  Endsley  emphasizes  three  components:  perception, 
comprehension  and  projection.  Perception  refers  to  recognition  of  the  assets  and  the 
elements  in  one’s  surroundings  and  their  relative  locations  and  states.  It  is  the  basic  state 
in  the  establishment  of  overall  SA.  Comprehension  is  the  next  state  in  setting  the  SA;  it 
refers  to  understanding  the  meanings  of  the  assets  and  elements  recognized  in  the 
perception  level.  For  example,  what  do  they  mean  to  a  pilot,  and  what  are  their  effects  on 
the  pilot’s  mission?  Comprehension  is  involved  in  creating  the  overall  picture  for  the 
pilot.  The  last  state  is  projection:  what  is  likely  to  happen  in  the  near  future,  who  and 
what  are  going  to  be  where,  what  are  the  possible  ways  that  the  elements  in  the  picture 
may  act 

From  the  pilot’s  view  of  point,  perception  is  realizing  and  identifying  other 
aircrafts,  threats,  terrain,  and  so  forth.  Comprehension  is  what  these  elements  mean  to  the 
pilot  and  how  they  might  affect  his  mission.  Projection  is  to  consider  what  the  possible 
courses  of  action  are  and  what  entities’  possible  courses  of  action  and  locations  will  be  in 
the  following  seconds/minutes/hours. 

A  different  example  might  help  the  reader  to  understand  the  terms  of  SA.  In 
football,  a  quarterback  (as  well  as  any  player  in  the  field,  but  especially  the  quarterback, 
since  he  can  be  taken  as  the  decision  maker  for  the  team  at  the  time  of  action)  needs  to 
identify  players  on  the  field  (both  on  his  team  and  the  other  team),  their  formation  on  the 
field,  the  perimeter  of  the  field,  etc.;  then  he  needs  to  understand  the  relative  importance 
and  capabilities  of  all  of  the  entities  (players,  ball)  he  has  identified;  finally,  he  must 
consider  the  possible  courses  of  action  of  all  the  entities  on  the  field  and  how  their  actions 
would  affect  his  decisions  and  actions,  as  well  as  how  his  decisions  and  actions  would 
affect  their  decisions  and  actions,  and  how  all  of  this  will  be  affected  by  where  all  the 
different  entities  will  be  in  the  future  as  a  result  of  the  possible  actions  taken. 

Using  the  definition  above,  Endsley  also  identifies  three  levels  of  failure 
associated  with  reduced  situation  awareness: 
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•  Level  1  -  Failure  to  correctly  perceive  information 

•  Level  2  -  Failure  to  correctly  integrate  or  comprehend  information 

•  Level  3  -  Failure  to  project  future  actions  or  state  of  the  system.16 

Any  work  on  SA  should  aim  to  eliminate  those  sources  of  failure  in  order  to 
enable  the  operator/pilot’s  ability  to  successfully  complete  the  mission.  An  interface  for 
an  SA  view  is  no  exception. 

Endsley  puts  forth  another  definition  that  is  simpler  and  more  basic.  “Situational 
awareness  (SA)  can  be  conceived  of  as  the  pilot’s  internal  model  of  the  world  around  him 
at  any  point  in  time.”17  However,  one  can  argue  that  this  definition  lacks  the  elements  of 
past  and  future  states,  such  as  projection,  that  were  contained  in  Endsley’ s  prior 
definition.  On  the  other  hand,  this  definition  includes  the  pilot  as  the  subject  and,  in  this 
aspect,  applies  to  the  approach  taken  in  this  thesis  more  than  the  other  definitions. 
Furthermore,  it  can  be  argued  term  of  internal  models  points  to  the  same  direction  of 
Dominguez’s  “coherent  mental  picture.” 

When  the  definition  is  viewed  from  the  eyes  and  the  mind  of  a  pilot,  some  other 
definitions  really  attract  a  pilot’s  attention  due  to  the  terminology  they  use.  One  of  these 
belongs  to  Eugene  C.  Adam.  In  his  work,  “Fighter  Cockpits  of  The  Future,”  Adam 
defines  SA  quite  simply  in  pilots’  terms:  “It’s  simply  KNOWING  WHAT’S  GOING  ON 
SO  YOU  CAN  FIGURE  OUT  WHAT  TO  DO!  Where  are  the  friendlies,  bogies,  SAMs 
and  unknowns  with  respect  to  my  flight?  What  are  their  intentions,  my  intentions  and  my 
options?”18 

In  his  work,  Adam  looks  at  SA  from  the  perspective  of  cockpit  design,  which  is 
similar  to  the  search  for  a  better  SA  view  for  the  Rascal  UAV  that  this  thesis  is 

16  Endsley,  “Toward  a  theory  of  situation  awareness  in  dynamic  systems.” 

17  Mica  R.  Endsley,  “Situation  awareness  global  assessment  technique  (SAGAT),”  in  Proceedings  of 
the  IEEE  1988  National  Aerospace  and  Electronics  Conference  (1988) 

http://ieeexplore.ieee.org/iel2/748/5021/00195097.pdf?tp=&amumber=195097&isnumber=5021)  (accessed 
November  20,  2007). 

1 8  Eugene  C.  Adam,  “Fighter  Cockpits  of  the  Future,”  Aerospace  and  Electronic  Systems  Magazine, 
IEEE  Volume  9,  Issue  3,  March  1994,  20  -  26, 

http://ieeexplore.ieee.org/iel3/62/6822/00275378.pdf?tp=&arnumber=275378&isnumber=6822  (accessed 

November  20,  2007). 
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undertaking.  He  tries  to  find  the  optimum  cockpit  design  to  give  the  pilot  the  best 
possible  SA  while  looking  at  his  instruments  and  indicators.  He  separates  SA  into  two 
parts:  “Global  and  Tactical.”  Global  -  also  known  as  “non  visual”  -  SA  has  a  range  of  up 
to  200  miles,  while  Tactical  SA  consists  of  the  visual  range.  Adam  attempts  to  find  a 
cockpit  design  that  will  optimize  utilization  within  the  cockpit,  and  will  enable  the  pilot 
to  gain  complete  SA  at  both  the  global  and  tactical  levels. 


SA  Has  2  Parts 


Figure  6.  Two  parts  of  SA  according  to  Eugene  C.  Adam19 

The  definition  of  SA  adopted  by  the  U.S.  Air  Force  blends  most  of  the  elements 
of  these  definitions  such  as  dynamism,  being  continuous,  affecting  the  decision  in  present 
and  future.  This  definition  is  as  follows:  “a  pilot's  continuous  perception  of  self  and 
aircraft  in  relation  to  the  dynamic  environment  of  flight,  threats,  and  mission  and  the 
ability  to  forecast  and  then  execute  tasks  based  on  that  perception.”20  One  may  claim  that 
this  definition  lacks  an  understanding  of  past  states  of  surrounding  elements,  but  it  can  be 
argued  that  such  an  understanding  is  included  within  the  understanding  of  the  dynamic 
nature  of  the  present. 


19  Adam,  “Fighter  Cockpits  of  the  Future.” 

20  Thomas  R.  Carretta,  David  C.  Perry,  Jr.,  and  Malcolm  James  Ree,  “Prediction  of  Situational 
Awareness  in  F-15  Pilots,”  The  International  Journal  of  Aviation  Psychology’,  6,  no.  1  (1996):  21-41 
http://www.leaonline.com/doi/pdf/lQ.1207/sl5327108iiapQ601  2  (accessed  November  20,  2007). 
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Although  there  are  variations,  all  the  definitions  still  point  in  the  same  general 
direction:  that  of  understanding  one’s  (the  decision  maker)  position  within  the 
surrounding  environment  surrounding  in  the  past  and  future  as  well  as  in  the  present.  This 
understanding  gives  the  decision  maker  better  capability  for  making  decisions. 

E.  DEFINITION  OF  INTERFACE 

For  any  system,  especially  any  Decision  Support  System  and  computer  system, 
acceptance  by  the  users  depends  highly  on  the  user  interface.  Simply  modifying  the 
words  of  Efraim  Turban,  the  key  to  successful  use  of  any  UAV  system  is  the  user 
interface.  The  simpler  the  use  of  the  system,  the  greater  the  chance  that  it  will  be  utilized 
by  the  operators.21  Users  cannot  judge  the  features  that  they  cannot  use.  If  an  interface  is 
not  “user  friendly,”  it  does  not  matter  how  good  the  system  and  features  are;  the  system  - 
and  the  business  that  makes  the  system  -  are  most  likely  destined  to  fail. 

In  his  book,  “Decision  Support  Systems  in  the  21st  Century,”  George  M.  Marakas, 
mentions  the  user  interface  as  one  of  the  five  core  components  of  Decision  Support 
Systems  (DSS).  His  five  core  components  are: 

•  “Data  management  systems 

•  Model  management  systems 

•  Knowledge  engines 

•  User  interface 

•  And  the  user”22 

Marakas  later  defines  user  interface  as  a  key  element  in  DSS  functionality.  He 
points  out  that  the  data;  model  and  the  processing  components  of  DSS  must  be  easily 
accessed  and  manipulated  to  support  the  decision  process  without  hindering  it.  He  also 
states  that,  ease  of  use  of  the  interface  to  communicate  with  DSS  while  executing  the 
functions  mentioned  is  vital  to  the  success  of  DSS.23 

21  Efraim  Turban,  Decision  Support  and  Expert  Systems:  Management  Support  Systems,  3rd  ed.  (New 
York:  Macmillan,  1998),  223. 

22  George  M.  Marakas,  Decision  Support  Systems  in  the  21st  Century >,  (New  Jersey:  Prentice-Hall, 
1999),  10-11. 

23  Ibid. 
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Although  the  term  “interface”  may  mean  an  interaction/communication  tool,  or  a 
layer  between  two  or  more  components  (ex.  steering  wheel,  throttle,  etc.),  or  have  other 
definitions  in  different  disciplines  ranging  from  chemistry  to  computer  science,24  the 
definition  of  interest  for  this  work  is  that  of  the  user  interface  in  computer  sciences, 
especially  the  Graphic  User  Interface  (GUI). 

The  tenn  “GUI”  goes  back  as  far  as  1945  in  theory,  but  its  first  practical 
application  was  achieved  by  the  work  of  Douglas  Engelbart  and  researchers  at  the 
Stanford  Research  Institute  in  XEROX  PARC  research  and  development  company  in  the 
1960s.  These  studies  arguably  paved  the  way  for  later  versions  of  GUI,  even  though  the 
tenn  would  not  come  into  popular  use  until  a  bit  later.25 

While  an  interface  is  a  layer  or  tool  for  interaction/communication  between 
components  in  general,  in  the  case  of  a  user  interface,  one  of  those  components  is  the 
user.  The  user  is  the  operator  of  the  system,  and  the  interface  provides  the  tools  and 
means  that  enable  the  user  to  operate  the  system  without  being  distracted  by  the  overhead 
process  load. 

Just  as  there  are  different  definitions  of  SA,  there  are  different  definitions  of  User 
Interface.  Most  of  those  points  to  interface  being  the  medium  between  the  user  and  the 
system  or  unit,  relieving  the  operator  of  knowing  all  the  complex  algorithms  and 
command  lines  by  providing  manageable  symbols  for  them.  Some  of  these  definitions 
follow. 

Marakas  defines  an  interface  as  “a  component  of  a  system  that  is  specifically 
intended  to  allow  the  user  to  access  the  internal  components  of  that  system  in  a  relatively 
easy  fashion  and  without  having  to  know  specifically  how  everything  is  put  together  or 
how  it  works  together.”26  While  Marakas  points  to  how  an  interface  saves  the  user  from 


24  Graphical  User  Interface  (GUI),  Britannica  Online  Dictionary, 
http://www.britannica.com/eb/article-91Q9589/graphical-user-interface  (accessed  November  28,  2007). 

25  Ibid. 

26  George  M.  Marakas,  Decision  Support  Systems  in  the  21st  Century ,  (New  Jersey:  Prentice-Hall, 
1999),  20. 
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the  burden  of  knowing  the  internal  structure  of  the  system,  he  carries  this  further  by 
saying,  “The  easier  it  is  for  a  user  to  access  the  system  the  better  the  interface.”27 

Turban  describes  the  interface  as  a  data  transportation  medium  between  user  and 
the  computer.  “The  user  interface  may  be  thought  of  as  a  surface  through  which  data  is 
passed  back  and  forth  between  user  and  computer.”28 

According  to  Turban,  GUIs  provide  visible  subjects  through  which  to  manipulate 
the  system,  while  hiding  the  complex  syntax  from  the  user  in  order  to  simplify  the 
manipulation.  He  says,  “GUIs  are  direct  manipulation  systems  in  which  the  users  have 
direct  control  of  visible  subjects  (such  as  icons),  and  actions  replace  complex  command 
syntax.  Users  just  touch  or  aim  at  visual  areas  to  interact  with  a  computer.”29 

Britannica  views  the  GUI  as  another  program  within  the  computer  that  allows 
communication  between  the  user  and  the  computer:  “[a]  computer  program  that  enables  a 
person  to  communicate  with  a  computer  through  the  use  of  symbols,  visual  metaphors, 
and  pointing  devices. ”30 

Like  Britannica,  Wordreference  online  source  points  to  GUI  as  an  interaction  tool 
between  the  user  and  the  computer:  “(computer  science)  a  program  that  controls  a  display 
for  the  user  (usually  on  a  computer  monitor)  and  that  allows  the  user  to  interact  with  the 
system”31 

Yet  another  definition  from  an  online  source  points  to  how  GUI  functions  in 
terms  of  operation  and  data  input:  “The  aspects  of  a  computer  system  or  program  which 
can  be  seen  (or  heard  or  otherwise  perceived)  by  the  human  user,  and  the  commands  and 
mechanisms  the  user  uses  to  control  its  operation  and  input  data.  A  graphical  user 


27  Marakas,  Decision  Support  Systems  in  the  21st  Century’. 

28  Efraim  Turban,  Decision  Support  and  Expert  Systems:  Management  Support  Systems,  3ld  ed.  (New 
York:  Macmillan,  1998),  224. 

29  Ibid.,  233-234. 

39  Graphical  User  Interface  (GUI),  Britannica  online  dictionary  article 
http://www.britannica.com/eb/article-91Q9589/graphical-user-interface  (accessed  November  28,  2007). 

31  Interface,  http://www.wordreference.com/defmition/interface  (accessed  November  28,  2007). 
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interface  emphasizes  the  use  of  pictures  for  output  and  a  pointing  device  such  as  a  mouse 
for  input  and  control  whereas  a  command  line  interface  requires  the  user  to  type  textual 
commands  and  input  at  a  keyboard  and  produces  a  single  stream  of  text  as  output.”32 

Merriam- Webster  points  in  the  same  direction,  but  also  includes  some  other 
aspects  that  were  missed  in  the  technicality  of  other  definitions  such  as  easiness  and 
making  choices:  “[a]  computer  program  designed  to  allow  a  computer  user  to  interact 
easily  with  the  computer  typically  by  making  choices  from  menus  or  groups  of  icons”33 
Even  though  these  two  aspects  are  apparent,  they  are  overlooked  in  other  definitions,  and 
this  may  lead  them  to  be  overlooked  in  design,  too. 

For  the  purposes  of  this  thesis,  a  combined  but  rather  simple  and  nontechnical 
definition  of  interface  especially  tailored  for  SA  views  was  developed.  According  to  this 
definition,  an  interface  is  a  decision  support  program  that  enables  the  user  to  interact  with 
a  UAV  system  and  make  decisions  for  its  operation.  By  doing  that,  an  interface  enables 
the  operator  to  execute  such  operations  easily  and  efficiently. 

F.  CONVERGENCE  OF  SA  AND  THE  INTERFACES:  SA  INTERFACES 

In  an  UAV  system,  just  like  in  any  system,  several  components  work  for  the 
overall  success  of  the  intended  mission.  While  all  of  them  exist  by  themselves  as  separate 
entities,  they  are  put  together  with  the  intention  of  creating  a  system.  The  main 
components  in  a  general  view  are: 

•  The  UAV  itself  flying  the  mission,  with  all  the  parts  making  up  the  flying 
object:  engine,  wings,  fuselage,  control  surfaces,  landing  gear,  etc.; 

•  Payload  for  the  mission(s),  which  also  defines  the  nature  of  the  mission(s): 
different  type  of  sensors,  armament,  exterior  payload  (medical,  food,  etc.), 
and  radio  communication  equipment; 

•  Control  stations  (ground  station,  airborne  station,  etc.); 

•  Takeoff,  landing  and  recovery  equipment  (if  applicable); 

32  Ami  Augmented  Multi-Party  Interaction  D.1.2  scenario  definition, 
http://209.85.!  73. 104/search?q=cache:-NDPPvZDF2MJ:www.  amiproiect.org/ami-scientific- 

portal/documentation/annual-reports/pdf/Dl  2.pdf  (accessed  November  28,  2007). 

33  Merriam-Webster  Online  Dictionary,  http://www.merriam- 
webster.com/dictionary/graphical%20user%20interface  (accessed  November  28,  2007). 
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•  Personnel  augmenting  the  flight  of  the  UAV  or  controlling  the  elements; 

•  Hardware  and  software; 

•  Last  of  all,  the  interface,  which  transforms  all  the  components  into  a 
system. 

One  can  add,  subtract  or  organize  the  given  list  of  components  differently  or 
include  any  of  those  components  within  another  one.  But  the  main  point  is  that  the 
interface  is  the  part  which  connects  all  of  the  components  and  makes  them  a  meaningful 
system  for  the  user’s  intended  mission(s),  In  this  case,  the  user  is  the  operator  of  the 
UAV.  All  the  capabilities  and  attributes  of  the  components  would  not  make  any  sense  in 
the  overall  picture  without  an  interface  that  enables  them  to  be  employed  within  the 
system  effectively.  From  the  SA  point  of  view,  a  UAV  system  exists  within  the 
environment  surrounding  it,  and  it  is  the  interface  component  that  enables  the  operators  to 
realize  their  position  and  condition  within  that  environment  by  connecting  the  elements 
of  the  system  for  the  user  visually  and  technically. 

In  a  vastly  broad  and  general  look,  especially  for  UAVs,  SA  interfaces  or  SA 
views  mainly  consist  of  a  display  of  the  area(s)  of  interest(s),  a  means  to  display  entities 
(including  UAV,  target(s),  enemy  and  friendly  forces,  etc.)  within  the  area  (mainly  by 
icons),  and  tool  boxes  either  surrounding  the  area  displays,  or  on  separate  screens 
designed  for  that  purpose.  In  general,  displays  of  the  area  employ  a  kind  of  map, 
previously-taken  area  photo,  graphics,  or  a  combination  of  these.  Such  displays  are  also 
known  as  geospatial  displays.34  Tool  boxes  surrounding  the  map  are  designed  to  help  the 
user  execute  the  mission  by  giving  commands  to  entities,  communicating  with  the 
entities,  changing  route,  defining  entities,  etc.  Figures  7  and  8  illustrate  two  of  the  sample 
SA  interfaces  (or  SA  views,  as  they  are  generally  referred)  referenced  frequently  in  this 
thesis.  It  is  possible  to  observe  the  components  mentioned  above  in  both  similar  and 
different  applications. 


34  Donelly,  et  at.,  “Commander’s  Intent  in  User  Interfaces  for  Network-Centric  Operations,”  (paper 
presented  in  12th  ICCRTS,  June  19-21,  2007) 

http://www.dodccrp.org/events/12th  ICCRTS/CD/html/papers/1 56.pdf  (accessed  February  10,  2008). 
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Figure  7.  Bullseye  SA  interface  (SA  view) 


18 


Figure  8.  SA  interface  used  for  the  Rascal  UAV  by  NPS 
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II.  BACKGROUND 


This  chapter  will  examine  the  history  of  unmanned  aviation,  as  well  as  the 
concepts  of  controlling  a  UAV  over  the  network  and  from  a  manned  aircraft.  The  first 
part  will  focus  on  the  history  of  UAVs  in  military  forces,  especially  the  U.S.  Armed 
Forces,  while  the  second  part  will  focus  on  the  concept  and  history  of  controlling  UAVs, 
especially  over  a  network. 

A.  BACKGROUND  ON  UAVS  AND  EMPLOYMENT  OF  UAVS 

An  unmanned  vehicle  is  defined  in  the  US  Secretary  of  Defense’s  Unmanned 
Systems  Roadmap  as  “[a]  powered  vehicle  that  does  not  carry  a  human  operator,  can  be 
operated  autonomously  or  remotely,  can  be  expendable  or  recoverable,  and  can  carry  a 
lethal  or  nonlethal  payload.  Ballistic  or  semi-ballistic  vehicles,  cruise  missiles,  artillery 
projectiles,  torpedoes,  mines,  satellites,  and  unattended  sensors  (with  no  form  of 
propulsion)  are  not  considered  unmanned  vehicles.  Unmanned  vehicles  are  the  primary 
component  of  unmanned  systems.”35  This  definition,  however,  was  not  the  concept 
utilized  throughout  the  history  of  Unmanned  Systems  and  UAVs.  A  number  of  vehicles 
that  would  not  be  considered  UAVs  according  to  the  above  definition  will  be  discussed  in 
order  to  show  the  progress  of  Unmanned  Aviation. 

The  beginnings  of  unmanned  aviation  can  be  tracked  back  almost  to  the 
beginnings  of  manned  flight  itself,  both  as  an  idea  and  as  an  operational  reality. 
Following  the  invention  of  the  hot  air  balloon  in  1783,  suggestions  were  made  that  they 
be  used  as  bombers,  by  either  powering  them  or  by  relying  on  the  wind  to  carry  them  to 
the  enemy’s  area  of  operations.  This  idea  was  turned  into  reality  during  the  siege  of 
Venice  by  the  Austrians  in  the  spring  of  1849. 36  Although  balloon  bombers  had  only 


35  Department  of  Defense,  Office  of  the  Secretary  of  Defense,  Unmanned  Systems  Roadmap  2007- 
2032,  1. 

3®  “Remote  Piloted  Aerial  Vehicles:  An  Anthology,  Remote  Piloted  Aerial  Vehicles,” 
www.ctie.monash.edu.au/hargrave/rpav  ome.html  (accessed  March  11,  2008). 
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minor  effects  on  the  siege  (they  were  turned  back  on  the  Austrians  by  the  wind),  this  still 
proved  that  remotely  operated  (or  initiated  in  this  case)  vehicles  could  be  used  in  a  battle. 

Even  though  balloons  did  not  have  a  big  impact  in  the  siege  of  Venice,  their  use 
there  inspired  others  to  make  use  of  them  in  later  battles.  For  example,  during  the 
American  Civil  War,  both  sides  used  balloons  loaded  with  explosive  devices  as 
unmanned  aerial  vehicles.  The  idea  was  to  cause  explosions  by  having  the  balloons  come 
down  inside  supply  or  ammunition  depots.37 

Studies  about  unmanned  vehicles  did  not  stop  between  wars.  In  the  initial  phases, 
balloons  were  studied,  largely  because  they  were  the  option  that  made  the  most  sense  in 
the  specific  circumstances  of  the  time,  given  that  aircraft  had  not  yet  matured  and 
wireless  technologies  were  not  readily  available.  During  WWI,  the  US  Army  worked  on 
developing  unmanned  wind-driven  balloons  to  be  used  as  bomb-carrying  attack  vehicles. 
Successful  program  prevented  by  the  annistice  to  be  operational  reality.38  During  WWI, 
the  US  Army  also  worked  on  a  flying  torpedo  known  as  the  “Kettering  Bug”  that  was 
inspired  by  the  Curtiss  Sperry  Aerial  Torpedo.39  The  Kettering  Bug  can  be  seen  as  more 
like  a  cruise  missile  than  a  UAV  but  still,  it  was  an  unmanned  flight.  During  about  the 
same  time  period,  the  British  worked  on  their  own  unmanned  systems,  producing  a 
device  that  was  able  to  fly  for  distances  of  up  to  300  miles.40 


37  Army  communicator,  “UAV  history,” 

http://www.gordon.army.mil/ac/Summer/Summer%20Q2/uav2.htm  (accessed  December  21,  2007). 

38  Charles  A.  Ziegler,  “Weapons  development  in  context:  The  case  of  the  World  War  I  balloon 
bomber,” 

http://www.istor.org/search/BasicSearch7sm  &hp=25&Search=Search&gw=itx&Query=Unmanned+air+v 

ehicles+history&wc=on  (accessed  November  11,  2007). 

39  “Unmanned  Aerial  Vehicles  and  Precision  Guided  Munitions  at  the  Centennial,” 
http://www.uavforum.com/library/defnews.doc  (accessed  November  11,  2007). 

40  Ibid. 
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Figure  9.  Kettering  Bug,  model  on  display  at  the  National  Museum  of  the  United 
States  Air  Force  in  Dayton,  Ohio41 

Between  WWI  and  WWII,  studies  of  unmanned  vehicles  continued.  However, 
around  the  1930s,  a  new  area  of  interest  in  unmanned  air  vehicles  developed.  While  work 
continued  on  the  use  of  unmanned  vehicles  for  bombardment  of  target  areas  (both  in  the 
form  of  using  the  vehicles  to  transport  bombs,  and  of  using  the  vehicles  themselves  as 
bombs),  another  branch  diverged  to  use  unmanned  vehicles  as  target  drones.  In  today’s 
concept,  those  vehicles  would  have  to  be  recoverable  in  order  to  be  considered  unmanned 
aerial  vehicles;  however,  rockets  may  still  be  considered  to  be  early  forms  of  unmanned 
vehicles,  since  they  mark  a  milestone  on  the  way  to  today’s  technology. 

During  WWII,  UAVs  saw  action  mainly  as  target  drones.  Both  the  United  States 
(US)  and  Japan  tried  to  use  balloons  and  modified  manned  aircraft  (Operation  Aphrodite) 
for  UAV  roles;  however,  those  projects  were  not  successful.42  Since  Joint  Publication  1- 
02 ’s  definition  states  that  vehicles  should  be  recoverable,  and  that  ballistic  or  semi- 
ballistic  vehicles,  cruise  missiles,  and  artillery  projectiles  are  not  considered  unmanned 
aerial  vehicles,  ,  German  V-l  and  V-2  rockets  would  not  be  considered  UAVs  in  today’s 
terms.  However,  they  should  be  mentioned  as  a  successful  step  toward  unmanned 
vehicles,  since  they  utilized  a  relatively  long  distance  unmanned  flight  to  reach  their 


41  Wikipedia,  “Kettering  Bug,”  http://en.wikipedia.org/wiki/Kettering  Bug  (accessed  February  10, 
2008). 


42 


Army  communicator,  “UAV  history.” 
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target  with  a  simple  form  of  guidance,  and  since  they  were  used  in  a  real  war  with 
relative  success.  In  addition,  they  were  the  first  example  in  which  unmanned  aircraft  were 
used,  thereby  protecting  the  lives  of  aircrews.43 

In  between  WWII  and  the  Vietnam  War,  the  defined  role  of  UAVs  (or  RPVs,  as 
they  were  referred  to  then)  was  as  target  drones.  This  target  drone  role  kept  UAV  works 
alive  for  the  most  part.  However,  target  drones  being  recoverable  led  naturally  to  the 
addition  of  a  reconnaissance  role  for  UAVs.  Adding  cameras  to  drones  gave 
reconnaissance  capability  to  target  drones  such  as  the  SD  series,  starting  with  the  SD-1 
Observer  (later  designated  as  MQM-57).44 

The  shooting  down  of  Gary  Power’s  U-2  on  1  May  1960  might  have  boosted 
work  on  UAV  reconnaissance  development  in  the  1960s,  but  anticipation  of  SR-71 
development  and  spy  satellite  programs,  as  well  as  President  Eisenhower’s  commitment 
to  end  reconnaissance  flights  over  of  the  Soviet  Union,45  hindered  this  possibility.  The 
Cuban  Crisis  provided  a  display  of  unmanned  reconnaissance  needs  that  led  to  the 
development  and  use  of  unmanned  platforms  in  Vietnam.46  In  Vietnam,  drones  modified 
from  their  target  drone  models,  such  as  the  Firebee  (AQM-34  series),  also  known  as  the 
Lightning  Bug,  were  used  as  UAVs,  mainly  in  reconnaissance  roles.  Between  20  August 
1964  and  30  April  1975,  those  UAVs  logged  3.435  operational  sorties  in  Southeast 
Asia.47 


43  “Unmanned  Aerial  Vehicles  and  Precision  Guided  Munitions  at  the  Centennial,” 
http://www.uavforum.com/librarv/defnews.doc  (accessed  February  10,  2008). 

44  Ibid.,  and  http://www.designation-svstems.net/dusrm/m-57.html  (accessed  February  10,  2008). 

45  Maj.  Christopher  A.  Jones,  USAF,  “Unmanned  Aerial  Vehicles  (UAVs):  An  Assessment  of 
Historical  Operations  and  Future  Possibilities”  (research  paper  presented  to  the  Research  Department,  Air 
Command  and  Staff  College,  March  1997). 

46  Ibid. 

47  Ibid.,  10. 
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Figure  10.  AQM-34;  US  Air  Force  Museum48 

During  the  late  1960s  and  early  1970s,  the  US  Navy  studied  the  possibility  of 
converting  the  Firebee  series  to  a  remote-controlled  anti-ship  missile  model,  and  the  US 
Air  Force  studied  the  possibility  of  converting  it  into  an  enemy  air-defense  suppression 
model  (with  successful  air  to  ground  AGM-65  Maverick  launches);  neither  of  these  led  to 
mass  production.49  The  end  of  the  Vietnam  War  also  marked  the  end  of  interest  in 
UAVs.50 


Figure  11.  AQM-34  with  Maverick  AGM51 

The  breakthrough  in  UAV  popularity  in  modern  times  began  in  1982,  after  The 
Arab-Israeli  war.  In  his  Joint  Force  Quarterly  essay,  Ralph  Sanders  even  names  UAVs  as 
an  Israeli  military  innovation.52 


48  Directory  of  US  military  rockets  and  missiles,  “Teledyn  Ryan  MQM-34,”  http ://www. designation- 
systems.  net/dusrm/m-34.html  (accessed  January  3,  2008). 

49  Ibid. 

59  Jones,  “Unmanned  Aerial  Vehicles.” 

51  Ibid.,  30. 

52  Ralph  Sanders,  “An  Israeli  Military  Innovation,”  Joint  Force  Quarterly  Winter  2002-03,  No.  33 
http://www.dtic.mil/doctrine/iefifq  pubs/2033. pdf  (accessed  February  10,  2008). 
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Employment  of  UAVs  (or  Remotely  Piloted  Vehicles:  RPVs)  in  reconnaissance 
and  electronic  countermeasure  roles  marked  a  milestone  in  modern  warfare  and  proved 
(or  even  created)  the  concept  of  combat  UAVs.  “Their  role  in  the  suppression  of  Syrian 
air  defense  was  described  as  a  “wake  up  call  for  the  US.”53 

During  the  1982  Arab-Israeli  war,  Syrian  SA-6  batteries  were  tricked  by  U.S.- 
designed  Samson  air-launched  decoys,  which  simulated  incoming  strike  aircraft.  While 
the  SA-6  batteries  responded  to  the  decoy  attacks,  Israel  deployed  Scout  and  Mastiff 
RPVs  for  surveillance  and  to  detect  the  frequencies  employed  by  the  SAM  radars.  Those 
frequencies  detected  by  the  RPVs  were  jammed  by  specially  configured  CH-53  Sea 
Stallion  helicopters,  while  Syrian  air  defense  forces,  fooled  by  the  decoys,  fell  victim  to 
Israeli  anti-radiation  missiles.  Massive  Israeli  air  strikes  and  long-range  artillery  support 
provided  ample  firepower,  while  Israeli  RPVs  monitored  the  battlefield.54 


Figure  12. 


1 


7tn 


MAZLAT  SCOUT 


MAZLAT  MASTIFF 


Scout  and  Mastiff  UAV 


53  Clifford  A.  Wright,  “The  Israeli  War  Machine  in  Lebanon  in  JSTOR,” 
http://links.istor.org/sici7sicU0377- 

919X%28198324%2912%3A2%3C38%3ATIWMIL%3E2.0.CO%3B2-G  (accessed  January  18,  2008). 

54  Major  G.D.  Thrash,  “USMC,  Remotely  Piloted  Vehicles-The  Unexploited  Force  Multiplier,”  by 
Major  G.D.  Thrash,  USMC,  CSC  1989,  http://www.globalsecuritv.org/inteMibrarv/reports/1989/TGD.htm 
(accessed  January  3,  2008). 
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Figure  13.  Scout  UAV55 


Following  the  operational  success  of  the  UAV,  interest  in  them  increased;  this 
interest  was  further  magnified  by  the  first  Gulf  War  and  operations  in  Kosovo.  Learning 
from  the  Israeli  experience,  and  also  motivated  by  identified  needs  from  conflicts  in 
Grenada  and  Libya,  the  US  Navy  started  the  Pioneer  program  in  the  1980s.  Originally  an 
Israeli  project,  the  Pioneer  program  became  a  joint  team  effort  between  Israel  and  the  US 
for  the  benefit  of  the  US  military.  UAVs  proved  themselves  valuable  ISR  assets  during 
Operations  Desert  Shield  and  Desert  Storm.56 


55  Iweapons,  www.israeli-weapons.com.  http://www.israeli- 
weapons.com/weapons/aircraft/uav/scout/Scoiit.html  (accessed  January  3,  2008). 

56  David  Glade,  “Unmanned  Aerial  Vehicles:  Implications  for  Military  Operations,”  (July  2000) 
http://www.au.af.mil/au/awc/awcgate/cst/occpprl6.doc  (accessed  February  10,  2008). 
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Figure  14.  Pioneer  UAV57 

This  capability,  along  with  added  roles  such  as  Combat  UAVs,  proved  the  value 
of  UAVs  in  Kosovo,  and  during  the  second  Gulf  War.  Successful  and  impressive 
performances  by  UAVs  in  these  operations  boosted  support  for  UAV  employment,58  and 
made  the  name  UAV  an  in-house  terminology  among  the  military  forces  of  the  world  as 
well  as  in  the  public  eye. 

Figure  15,  a  table  from  the  Unmanned  Systems  Roadmap,  summarizes  the  history 
and  progress  of  UAVs  in  the  US  Armed  Forces. 


57  John  Pike,  “Pioneer  Short  Range  (SR)  UAV,”  http://www.fas.org/irp/program/collect/pioneer.htm 
(accessed  February  10,  2008). 

58  Ibid.,  24. 
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I  AS  Role: 

Bneade  division  asset  for  RSTA 

Proponent 

Annv.  Manne  Corps 

Heritage: 

Falconer  ( 1950-60s)  -  Aquila  (1970-80s)  -  Pioneer  (1 980-2000s)-Dragon  Drone 

1 1990s)  -  Outrider  (1990s)  -  Shadow  200  (2000s) 

UAS  Role: 

Shipbome  asset  for  reconnaissance  and  weapon  support 

Proponent 

Naw 

Hen  rage: 

DASH  (1960s)  -  Project  Blaekflv  (1970s)-  Pioneer  (1980-2000s)-  Fire  Scout 
(2000s) 

UAS  Role: 

Small  unit  asset  for  over-the-hill  reconnaissance 

Proponent: 

Manne  Corps 

Hentage 

Bikini  (1960s)  -  Pointer  (1980-90*)  -  Dragon  Eve  (2000s) 

UAS  Role: 

Sumvable  asset  for  strategic  penetrating  reconnaissance 

Proponent 

Annv  Air  Force  Navy 

Heritage: 

Osprev  (1960s) -D -21  (1960s)  -  Classified  Program  (1980s)  -  DarkS tar  (1990s)- 
JUCAS  COOOs) 

UAS  Role: 

High  altitude  endurance  asset  for  standoff  reconnaissance 

Proponent: 

Air  F  orce 

Hentage: 

Compass  Arrow  (1960s)  -  Compass  Dwell  (1970s)  -  Compass  Cope  (1970s)  — 

Condor  (1980s)  -  Global  Hawk  (1 990-2000s) 

Figure  15.  Historically  Validated  UAS  Roles59 


Currently,  the  US  military  considers  UAVs  a  better  fit  for  “dull,  dirty,  or 
dangerous  missions  than  manned  aircraft”60  and  justifies  this  idea  by  stating,  “The 
attributes  that  make  the  use  of  unmanned  preferable  to  manned  aircraft  . . .  are,  in  the  case 
of  the  dull,  the  better  sustained  alertness  of  machines  over  that  of  humans  and,  for  the 
dirty  and  the  dangerous,  the  lower  political  and  human  cost  if  the  mission  is  lost,  and 
greater  probability  that  the  mission  will  be  successful.  Lower  downside  risk  and  higher 
confidence  in  mission  success  are  two  strong  motivators  for  continued  expansion  of 
unmanned  aircraft  systems.”61  The  roles  designated  for  present  and  future  UAVs  in  the 
Department  of  Defense’s  most  recent  Roadmap  for  UAVs  are  given  in  Figure  16.  Roles 
ranging  from  firefighting  to  air  warfare  to  mine  warfare  are  predicted.  Figure  17  shows 
the,  roadmap  to  achieve  the  vision  that  is  found  in  UAV  Roadmap  2007-2032. 


59  Department  of  Defense,  Office  of  the  Secretary  of  Defense,  Unmanned  Systems  Roadmap,  2007  - 
2032,  http://www.fas.org/irp/program/collect/usroadmap20Q7.pdf  (accessed  March  12,  2008). 

60  Department  of  Defense,  Office  of  the  Secretary  of  Defense,  Unmanned  Aircraft  Systems  Roadmap, 
2005  -  2030,  http://www.fas.org/irp/program/collect/uav  roadmap2Q05.pdf  (accessed  March  12,  2008). 

61  Ibid. 
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Figure  16.  Present  and  Future  Roles  for  DoD  Unmanned  Systems62 


6-  Department  of  Defense,  Unmanned  Systems  Roadmap,  2007-2032, 
http://www.fas.org/irp/program/collect/usroadmap2Q07.pdf  (accessed  March  12,  2008). 


30 


RECONNAISSANCE 


2005 

I 


2010 


2025 


2030 


2015 


2020 


TACTICAL  STRIKE 


COMM!  RELAY 


MARITIME  PATROL 


lIGBfT  COLLECTION 


SURVCH.LAMCC  t  BATTLE  MGMT 


AERIAL  REFUELING 


INTEGRATED  STRIKE  !  !EA0 
PENETRATING  STRIKE  ^ 


INT  STRIKE/SE AOlC  -  AM 


COUNTER  AIR 


|br  •  IILPOTOE  SURVtlL 


10/  ftrvtftMt 


MARITIME  INTERDICT  OP! 

I 

SURFACE  WARFARE 

m:m  jeaackiiweep 

A! W  ASA  M3 


ARMED  A  SA'iMS'  !U  A 


ELATION 


TELf  OWfUTED  COO 


JUTONGM^US  MAPPING  |  OCEANOGRAPHY  j  Aj, 


jJS 


IMVI UU  BH 


iSR 


K  A  TRAL. 


SUB  TR 


NJ 


vswiccoN 


MAN  PCM  •  AfiuE  CNJ 


IB  MB 


VMM  -  R1  1  --  i  -  | TR  •  r 


INSPECTION  >  O 

ncnciiSrwj 


mrORMATKIN  OP! 


EWTWNART  FACILITY  PATROL  A 


-  -*•  T:  *. 


^NAMPC^TABLE  EOO 

M  SSI.  ^^^Bri^^^nTM?A^^RAARrOFprtTK  F  -  I 


RANGE  CLEAR  A  CMiRUPT  LGi-L'tHFCl  E  IE0! 


'1  . 1 


OLLABORATTVE  COMBMlED  ARM! 


f  ME  NT 


Figure  17.  Joint  Services  Roadmap  for  Achieving  DoD  Vision  for  Unmanned 
Systems63 


As  can  be  observed  from  these  figures,  the  future  for  UAVs  holds  more  roles  and 
responsibilities  than  the  basic  reconnaissance  and  limited  capability  airborne  gun 
platforms.  UAV  technologies  and  concepts  can  be  expected  to  only  mature,  and  the 
concept  studied  by  Landreth  and  Glass  and  the  like  paves  the  road  for  this  maturing 
process. 


B.  BACKGROUND  ON  CONTROLLING  UAV  OVER  THE  NETWORK  AND 

SOME  CURRENT  EXAMPLES 


Although,  balloons  are  mentioned  above  as  the  first  unmanned  vehicles  in  basic 
terms,  they  were  not  really  controllable  vehicles,  since  they  were  bound  by  wind 
direction.  The  roots  of  successful  wireless  control  can  be  traced  back  to  1893,  to  St. 

63  Department  of  Defense,  Unmanned  Systems  Roadmap,  2007-2032. 
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Nicola  Tesla  and  his  telautomatons  (robots);  Tesla  further  demonstrated  remote  control  of 
objects  by  wireless  in  an  exhibition  in  1898  at  Madison  Square  Garden.  Then  Gabet  and 
his  ’Torpille  Radio- Automatique',  by  1909,  demonstrated  the  use  of  wireless  control 
following  wireless  controlled  aircrafts  or  with  more  popular  and  appropriate  name  R/C 
planes.64 

In  general,  UAV  control  is  executed  by  direct  Line-Of-Sight  (LOS)  or  via  satellite 
link  Beyond-Line-Of-Sight  (BLOS).  However,  network  technologies  and  a  special 
branch  of  networks  known  as  Mobile  Ad  Hoc  Networks  (MANET),  inspired  the  idea  of 
controlling  UAVs  over  the  networks  via  mobile  network  nodes.  These  network  nodes  can 
be  the  ground  control  station  launching  and  recovering  the  UAV,  forward  deployed  teams 
controlling  the  UAV  for  the  specified  mission  in  the  target  area,  and  the  manned 
aircraft(s)  controlling  the  UAV  from  launch  site  to  target  area  and  flying  the  mission  in 
the  target  area  if  needed. 

The  technology  to  establish  a  MANET  and  control  the  UAV  over  the  network 
already  largely  exists,  which  enables  the  military  to  gain  BLOS  capability  with  LOS 
technology.  Landreth  and  Glass’s  thesis  dealt  with  the  concept  of  controlling  UAVs  over 
the  network.  As  discussed  in  some  detail  in  Chapter  I,  their  thesis  proved  that  the  concept 
works.  There  are  also  some  other  studies  on  the  same  concept,  which  one  of  them  will  be 
discussed  below. 

One  study  was  done  under  Office  of  Naval  Research-Small  Business  Technology 
Transfer  ONR-STRR  program  jointly  by  Nascent  Technology  Corporation  and  MIT.  This 
study  developed  a  slightly  different  way  of  controlling  UAVs  over  the  network.  Within 
this  program,  BLOS  command  of  an  unmanned  helicopter  (away  helicopter)  is  achieved 
by  using  another  helicopter  (relay  helicopter)  as  a  relay  to  maintain  the  link  between  the 
tactical  center  and  the  away  helicopter.  Using  a  UAV  as  an  airborne  relay  node  is  not  a 


64  “Remote  Piloted  Aerial  Vehicles:  An  Anthology,  Remote  Piloted  Aerial  Vehicles,” 
http://www.ctie.monash.edu.au/hargrave/rpav  home.html  (accessed  January  5,  2008). 
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new  concept;  in  the  case  of  the  Hunter  Joint  Tactical  UAV,  the  radius  of  action  could  be 
increased  from  200  km  to  300  km  by  using  another  Hunter  as  an  airborne  relay.65 

Figure  18  displays  actual  flight  data  of  the  two  helicopters  during  demonstrations 
on  August  22,  2005.  In  the  figure,  the  red  circles  are  the  'away'  helicopter,  which  is 
navigating  to  the  point  marked  with  the  red  star,  and  the  green  circles  are  the  'relay' 
helicopter.66 
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Figure  18.  Actual  flight  data  of  the  two  helicopters  in  the  demonstration  of  ONR- 
STTR67 


65  David  Glade,  “Unmanned  Aerial  Vehicles:  Implications  for  Military  Operations,”  (July  2000) 
http://www.au.af.mil/au/awc/awcgate/cst/occpprl6.doc  (accessed  February  10,  2008). 

66  Nascent  Technology  Corporation,  “August  22,  2005:  Two  helicopters  demonstrate  beyond  line-of- 
sight  mission,”  http://www.nascent-tech.coip/press.htm  (accessed  January  2008). 

67  Ibid. 
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6^  Nascent  Technology  Corporation,  “August  22,  2005:  Two  helicopters  demonstrate  beyond  line-of- 
sight  mission.” 
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III.  PROBLEMS  EXPERIENCED  IN  CURRENT  AND  PAST  SA 

INTERFACES 


A.  SAMPLE  INCIDENTS  OF  MISHAPS  DUE  TO  SA  INTERFACE  DESIGN 

FAULTS  OR  SA  LOSS  OF  THE  OPERATOR/PILOT 

This  section  will  discuss  some  SA  and  interface  related  mishaps.  These  examples 
are  from  USAF  Accident  Investigation  Board  (AIB)  Class  A  report  summaries,  and  from 
personal  observations  of  a  TNT  experiment  on  November  2007.  These  examples  point  to 
the  kind  of  results  that  can  be  created  by  faulty  or  deficient  interface  designs. 

One  of  the  mishaps  is  taken  from  USAF  accident  reports  for  MQ-1B  Predator  S/N 
03-3112  on  January  17,  2007.  This  mishap  demonstrates  interface  deficiency  in 
continuously  monitoring  critical  engine  parameters  and  displaying  engine  indication 
deficiencies  to  the  UAV  operator.  One  reason  this  mishap  was  selected  as  an  example  is 
because  the  current  SA  view  of  the  Rascal  UAV  used  in  TNT  experiments  lacks  any  kind 
of  alert  mechanism,  especially  any  alert  regarding  Rascal  engine  parameters  such  as 
RPM. 


Approximately  14  hours  into  a  20  hours  sortie,  the  aircraft  sustained  a 
momentary  (two  (2)  seconds)  drop  in  engine  rotations  per  minute  (RPM) 
followed  15  minutes  later  by  catastrophic  engine  failure.  Data  logger 
analysis  of  changes  in  RPM,  oil  pressure,  turbo  oil  temperature,  and 
propeller  pitch  subsequent  to  original  two  second  RPM  drop  indicate  the 
engine  was  failing  over  a  period  of  15  minutes.  However,  monitored 
engine  parameters  remained  within  nonnal  ranges  until  approximately  the 
last  minute  before  the  engine  seized.  Therefore,  Mishap  Aircraft  (MA)  did 
not  generate  any  fonn  of  caution  or  warning  to  the  pilot  of  the  impending 
failure  until  approximately  60  seconds  prior  to  the  engine  completely 
failing.69 


69  Executive  summary  aircraft  accident  investigation  MQ-1B  Predator  S/N  03-3 1 12. 
http://usaf.aib.law.af.mil/  (accessed  January  17,  2007). 
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One  other  example  was  observed  during  TNT  experiments  at  Camp  Roberts,  CA 
during  November  2007.  The  author  of  this  thesis  was  observing  the  operation  of  a 
“Raven”  UAV  from  the  control  and  operation  site  at  the  time  of  mishap.  In  the  author’s 
judgment,  this  simple  mishap  was  caused  by  SA  interface-related  pilot  SA  deficiency. 

On  November  6,  2007  at  09:09  am,  a  Raven  UAV  operated  by  contractor 
WINTEC  flew  into  a  tree  while  executing  its  target  search  and  acquisition  mission  for 
suspected  vehicles  under  an  Aerial  Search  Optimization  Model  (ASOM)  scenario.  In  the 
case  of  this  specific  UAV,  basic  operation  is  executed  by  a  team  of  two  personnel:  one 
mission  controller  and  one  pilot.  Due  to  the  SA  interface  design,  the  pilot  has  a  limited 
SA  since  he/she  has  only  the  video  feed  to  with  which  to  navigate,  and  that  video  feed  is 
limited  to  the  camera  Field  of  View  (FOV)  video  feed  without  an  area  map  aid.  Because 
the  Raven  either  completely  lacks  or  has  only  a  limited  forward  (tilted  downward) 
looking  camera,  the  pilot  cannot  really  see  obstacles  coming  at  the  UAV  throughout  the 
route.  Thus,  the  pilot  is  bound  to  the  mission  controller  for  obstacle  avoidance  in  low 
altitude  flight. 

During  the  time  of  the  mishap,  the  mission  controller  became  distracted  when  he 
needed  to  communicate  with  other  UAV  personnel  flying  the  Buster  UAV  in  the  same 
scenario  from  the  same  launch  and  control  site.  For  future  reference,  this  might  be  a  good 
simulation  exercise,  since,  in  real  operational  conditions  out  in  the  field,  a  mission 
controller  and  an  operator  cannot  expect  to  execute  their  mission  in  perfectly  ideal 
conditions.  Distractions  should  be  an  expected  operational  condition  in  real  life,  just  as  it 
happened  in  this  experiment.  In  this  phase  of  the  mission,  the  Raven  was  flying  from  one 
target  area  to  another  and  no  specific  event  was  expected,  so  both  personnel  were 
relatively  relaxed.  While  the  mission  controller  was  communicating  with  the  other 
personnel,  the  Raven  flew  into  a  tree  in  ascending  terrain  in  low  altitude  flight;  the  pilot 
was  not  able  to  see  the  obstacles  in  front  of  the  UAVs  flight  path,  and  the  mission 
controller  was  engaged  with  other  mission  related  operations.  At  the  time  of  the  mishap, 
the  Raven  had  been  airborne  for  approximately  75  minutes.  The  absence  of  a  low  altitude 
warning  the  pilot’s  limited  SA  and  his  dependence  on  the  mission  controller  for  basic  SA 
all  contributed  to  the  mishap. 
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The  mishap  did  not  affect  the  experiment,  since  there  were  spare  Raven  UAVs  at 
the  site.  The  Raven  that  was  involved  in  the  mishap  was  recovered  shortly  with  no 
significant  damage,  and  continued  flying  during  the  day  without  any  further  problems. 


Figure  20.  Raven  prepares  to  launch70 


F igure  2 1 .  Buster  U A V7 1 


70  Directory  of  US  military  rockets  and  missiles,  “AeroVironment  RQ-1 1  Raven” 
http://www.designation-svstems.net/dusrm/app2/q-l  l.html  (accessed  November  10,  2007). 

71  Directory  of  US  military  rockets  and  missiles,  “Mission  Technologies  Buster,” 
http://www.designation-svstems.net/dusrm/app4/buster.html  (accessed  November  10,  2007). 
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B. 


UAV  INTERFACE  PROBLEMS 


This  section  will  briefly  examine  some  of  the  problems  identified  with  different 
kinds  of  UAVs  and  their  respective  interfaces,  with  an  eye  toward  identifying  some  of  the 
general  areas  of  dissatisfaction  for  the  users.  This  will  be  followed  by  an  examination  of 
the  interface  used  by  the  NPS  TNT  team  for  the  Rascal  UAV. 

Any  computer  or  machine  interface  may  be  the  source  of  some  sort  of  user 
dissatisfaction  and  eventually  may  lead  to  an  accident  or  mishap.  However,  UAV 
interfaces  have  their  unique  aspects: 

•  Interfaces  are  created  by  people  with  limited  UAV  control  experience: 

•  This  problem  is  exaggerated  by  the  fact  that  the  user  and  the  machine  are 
located  far  away  from  each  other,  hundreds  of  miles  in  some  cases; 

•  Machine  (UAV)  feedbacks  to  the  user  (pilot,  operator)  are  limited.  One 
UAV  operator  described  this  by  saying,  “It’s  like  driving  your  car  with 
paper  towel  tubes  over  your  eyes. .  .;72 

•  The  duration  and  automated  nature  of  the  missions  create  another  problem 
area  for  UAV  missions; 

•  The  relative  immaturity  of  the  UAV  and  combat  UAV  concepts  bring 
some  additional  problems. 

As  was  mentioned  in  Chapter  I  when  SA  was  defined,  decision  makers  need  a 
healthy  and  correct  information  flow  in  order  to  make  healthy  decisions.  The  current 
condition  can  be  compared  to  a  runner  running  as  fast  as  he  can  in  complete  darkness  that 
is  illuminated  only  by  a  very  limited  and  narrow  flashlight  beam,  while  simultaneously 
trying  to  make  fast  and  accurate  decisions  and  trying  to  identify  features  and  items  in  the 
dark  environment.  As  soon  as  the  flashlight  is  turned  in  a  different  direction,  the 
previously  illuminated  part  turns  into  complete  darkness.  In  a  research  experiment  on 
ground-based  robots,  it  was  found  that  “search  and  rescue  workers  participating  in  a 
ground-based  robot  experiment  spent,  on  average,  approximately  30%  of  the  time  solely 
trying  to  gain  or  maintain  SA,  which  chiefly  consisted  of  understanding  their  remote 

72  Mark  H.  Draper,  “UAV  Operator  Interface  Research  at  AFRL/HEC,”  (May  24,  2004)  presented  in 
2004,  Cerici  workshop, 

http://www.cerici.com/documents/2004  workshop/UAVHumanF actorsResearch.pdf  (accessed  January  13, 
2008). 
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robot’s  location,  surroundings,  and  status.”73  This  implies  that  important  time  is  lost 
while  the  operator  tries  to  understand  the  asset’s  position  instead  of  executing  the 
mission. 

Another  problem  area  results  from  the  duration  and  highly  automated  nature  of 
the  missions,  which  makes  monitoring  the  UAV  flight  bothersome  at  the  least.  While 
piloting  involves  feeling  the  aircraft  and  seeing  the  environment  surrounding  it,  UAV 
piloting  lacks  the  classic  feelings  involved  in  flying  an  aircraft:  no  more  feeling  the  buffer 
on  the  wingtips,  hearing  a  strange  sound,  smelling  something  funny  in  the  aircraft,  feeling 
the  “G”  load,  or  feel  of  the  pants. 

One  more  factor  increases  the  potential  for  problems.  The  UAV  concept  is  a 
rather  new  concept  in  popular  use.  Moreover,  newer  usage  concepts  continue  to  arise,  and 
the  designers  of  the  interfaces  for  those  new  concepts  and  the  UAVs  are  generally  not  the 
actual  users  in  real  conditions.  It  is  as  if  the  concept  was  crawling  for  years  and  then 
started  to  run  in  a  couple  of  hours,  and  the  body  is  having  a  hard  time  keeping  up  with 
this  sudden  change.  A  decrease  in  these  problems  will  happen  eventually  as  experience  in 
implementation,  improved  techniques,  and  better  products  are  developed.  This  change 
has  already  begun,  as  can  be  seen  from  the  graphs  for  the  Predator  UAV,  below  Figure 
22. 

When  one  examines  the  number  of  mishaps  per  flight  hour  from  their  first 
inception,  the  numbers  for  UAVs  look  discouragingly  bad;  for  example,  “the  Air  Force's 
RQ-1  Predator  accumulated  a  mishap  rate  of  32  mishaps  per  100,000  flight  hours,  the 
Navy /Marine's  RQ-2  Pioneer  334  mishaps  per  100,000  hours,  and  the  Army's  RQ-5 
Hunter  55  mishaps  per  100,000  hours”74  But  these  numbers  might  not  represent  the 
whole  truth.  Even  though  the  number  of  mishaps  increased  over  the  years,  the  increase  in 
number  of  the  hours  caused  to  accident  per  100,000  hours  ratio  to  decrease.  The  ratio  of 

73  Drury,  et  at.,  “Comparing  Situation  Awareness  for  Two  Unmanned  Aerial  Vehicle  Human  Interface 
Approaches,”  http://stinet.dtic.miPcgi- 

bin/GetTRDoc?AD=ADA456256&Location=U2&doc=GetTRDoc.pdf  (accessed  February  4,  2008). 

74  William  T.  Thompson,  Stefan  H.  Constable  and  Anthony  P.  Tvaryanas,  “U.S.  Military  Unmanned 
Aerial  Vehicle  Mishaps:  Assessment  of  the  Role  of  Human  Factors  Using  Human  Factors  Analysis  and 
Classification  System  (HFACS),”  http://stinet.dtic.mil/cgi- 

bin/GetTRDoc?AD=ADA435063&Location=U2&doc=GetTRDoc.pdf  (accessed  November  10,  2008). 
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mishaps  per  100,000  flying  hours  is  actually  approaching  the  manned  aircraft  level  of  one 
per  100,000  flying  hours.  However,  interface  related  mishaps  still  stand  as  an  important 
factor.  According  to  some  sources,  more  than  half  of  the  UAV  mishaps  are  attributed  to 
interface  designs.75 

At  this  point,  a  closer  look  at  the  given  figures  can  help  in  understanding  of  the 
trend  in  UAV  mishaps.  Figure  22  demonstrates  the  decrease  in  mishap  numbers  as  the 
flight  hours  increase.  It  should  be  noted  that  some  of  the  UAV  mishap  rates  are  already 
under  the  manned  F-16  class  A  mishap  rate,  and  the  ones  above  it  are  showing  decreasing 
trends. 


Figure  22.  Class  A  Mishap  Rates  (Lifetime),  1986-200676 


75  Drury,  et  al.,  “Comparing  Situation  Awareness  for  Two  Unmanned  Aerial  Vehicle  Human  Interface 
Approaches.” 

76  Department  of  Defense,  Unmanned  Systems  Roadmap,  2007-2032, 
http://www.fas.org/irp/program/collect/usroadmap2QQ7.pdf  (accessed  March  12,  2008). 
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Figure  23  gives  a  comparative  graph  of  mishaps  during  initial  50,000  flight  hours. 
While  F-16  and  U-2  followed  a  widely  used  and  accepted  manned  aircraft  experience, 
UAVs  are  relatively  new  entities  with  new  technologies,  techniques  and  implementations 
(including  interfaces).  Figures  24  through  28  strengthen  the  view  that,  as  the  flight  hours 
increase,  the  mishap  rate  decreases,  which  is  quite  rational  since  more  experience  and  the 
correction  of  initial  mistakes  in  design  and  operating  procedures  should  result  in  better 
products  and  procedures. 


During  initial  50,000  flight  hours 
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Figure  23.  Mishap  rate  in  initial  50,000  hours77 


77  Bob  Nullmeyer  and  Lt  Col  Robert  Herz,  “Predator  Decision  Making  and  Human  Error:  Implications 
for  training  21st  Century  Warriors,”  http://www.cerici.com/documents/2007  workshop/4-3-AFRL-Herz.pdf 
(accessed  January  13,  2008). 


41 


to 


First  Half  Mishap  Human  Factors 

(1997-2003) 


8  - 

7  • 

8 

Number  x  of 
Class.  A 
Mishaps  4  . 

3  - 

2 

1  ■ 

0 


□  Major 

□  Causal 


/  /  ^  ^  < 


N-g 

O]>efaltoo» 

Mishaps 


c r 


<g»  o 

Satires  Safety  Center  Human  Factors  Catabase 


Figure  24.  Predator  97-03  First  Half  Mishap  Human  Factors78 


Second  Half  Mishap  Human  Factors 

(2004-2006) 


Figure  25.  Predator  04-06  Second  Half  Mishap  Human  Factors79 


78  Bob  Nullmeyer  and  Lt  Col  Robert  Herz,  “Predator  Decision  Making  and  Human  Error:  Implications 
for  training  21st  Century  Warriors.” 

79  Ibid. 
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Figure  26.  Predator  Class  A  Mishap  Frequencies  and  Causes80 
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Figure  27.  Predator  hours  flown81 


80  Robert  T.  Nullmeyer,  Lt  Col  Robert  Herz,  Gregg  A.  Montijo  and  Robert  Leonik,  “Birds  of  Prey: 
Training  Solutions  to  Human  Factors  Issues,”  http://www.iitsec.org/documents/HP.pdf  (accessed  January 
13,2008). 

81  Ibid. 
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Predator  Class  A  Mishap  Rates 


© 


FY99  FYOO  FYOI  FY02  FY03  FY04  FY05  FY06 
Source:  vww  etsattlv  at  mi  Fw»(yv«MPr«t«<») 

Figure  28.  Predator  Class  A  Mishap  Rates82 

Another  source  of  problems  resulting  from  the  newness  of  the  UAV  concept  is  a 
lack  of  commonness  between  user  interfaces.  Even  though  there  is  an  effort  underway  by 
the  likes  of  NATO  STANAG  4586  “Standard  Interface  of  the  Unmanned  Control  System 
(UCS)  for  NATO  UAV  interoperability”  to  standardize  UAV  interfaces,  it  is  as  yet  far 
from  being  common.  As  the  armed  forces  of  multiple  countries  either  employ  UAVs  for 
their  missions  or  plan  to  develop  new  UAVs  for  future  missions,  as  is  seen  in  the 
example  of  the  US  Department  of  Defense’s  Unmanned  Aircraft  Systems  Roadmap,  they 
realized  that  there  was  a  possibility  of  using  similar  systems  within  their  own  structure 
and  with  foreign  forces.  Possibly  with  cost  reduction  and  interoperability  in  mind,  they 
decided  to  develop  NATO  STANAG  4586  “to  decouple  the  control  station  from  the  air 


82  Bob  Nullmeyer  and  Lt  Col  Robert  Herz,  “Predator  Decision  Making  and  Human  Error:  Implications 
for  training  21st  Century  Warriors,”  http://www.cerici.com/documents/2007  workshop/4-3-AFRL-Herz.pdf 
(accessed  January  13,  2008). 
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vehicle  in  UAV  systems,  thus  presenting  the  option  of  developing  a  single  control  station 
to  control  a  range  of  UAV  types  in  disparate  missions  and  also  to  allow  for  greater 
interoperability”83  Even  though  it  is  a  step  forward,  NATO  STAN  AG  4586  has  its  own 
problems  at  this  stage,84  and  is  not  a  magical  complete  solution  to  Human  Computer 
Interface  (HCI)  problems. 

As  Marakas  states,  as  the  interfaces  became  more  common,  efforts  to  use  them 
and  training  efforts  to  learn  a  system  between  moving  one  system  to  another  will 
decrease.85  So  will  the  costs  related  to  developing  a  user  interface,  problems  due  to 
design  deficiencies,  and  the  problems  arising  from  this  transformation. 

One  of  the  detailed  available  user  reports  comes  from  the  2001  Global  Hawk 
deployment  to  Australia.  In  this  deployment,  Mission  Control  Element  (MCE)  operators 
rated  status  displays  and  controls  in  the  MCE  as  consistently  unacceptable.  The  table  in 
Figure  29  lists  the  operator  ratings  for  the  Global  Hawk  UAV. 


83  M.L.  Cummings,  J.T.  Platts,  A.  Sulmistras,  “Human  Performance  Considerations  in  the 
Development  of  Interoperability  Standards  for  UAV  Interfaces,”  presented  at  the  Moving  Autonomy 
Forward  Conference,  2006. 

84  Ibid. 

85  Marakas,  “Support  Systems  in  the  21st  Century.” 
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Question 

Median  Rating 

Rate  the  mental  effort  workload  during  task  completion  with 
the  Global  Hawk  (GH)  UAV  system. 

Poor  to  Acceptable 

Rate  the  physical  workload  during  task  completion  with  the 

GH  UAV  system. 

Acceptable 

Rate  the  time  pressure  during  task  completion  with  the  GH 

UAV  system. 

Acceptable 

Rate  the  work  backlog  during  task  completion  with  the  GH 

UAV  system. 

Acceptable 

Rate  the  adequacy  of  tire  physical  arrangement  of  the 
workspaces  in  the  MCE. 

Unacceptable  to  Poor 

Rate  the  lighting  in  the  MCE  facility. 

Acceptable 

Rate  the  heating  and  cooling  facilities  in  the  MCE. 

Poor  to  Acceptable 

Rate  the  level  of  noise  in  the  MCE. 

Acceptable 

Rate  the  freedom  of  motion  within  the  MCE 

Poor 

Rate  the  utility  of  the  status  displays  and  indicators  in  the 

MCE. 

Unacceptable 

Rate  the  utility  of  the  controls  and  menus  in  the  MCE. 

U  nacceptable 

Rate  the  ease  of  access  to  displays,  indicators,  and  controls  in 
the  MCE 

Unacceptable 

Figure  29.  Mission  control  element  MCE  Operators’  responses  to  Human  Factors 
Questionnaire86 

Furthermore,  some  of  the  areas  are  identified  as  problematic,  such  as: 

•  physical  arrangement  of  the  displays  (too  far  apart); 

•  the  unnecessarily  complicated  retasking  processes; 

•  difficult-to-read  displays  (due  to  the  fonts  and  colors  that  were  used); 

•  difficulty  of  monitoring  the  automated  system  closely  over  extended 
periods; 

•  no  access  to  the  vestibular  cues  that  pilots  of  manned  aircraft  use  to  gain 
an  understanding  of  the  orientation  of  the  aircraft; 


86  Robyn  Hopcroft,  Eleanore  Burchat  and  Julian  Vince,  “Unmanned  Aerial  Vehicles  for  Maritime 
Patrol:  Human  Factors  Issues,”  http://www.dsto.defence.gov.au/publications/4589/DSTO-GD-Q463.pdf 
(accessed  January  19,  2008). 
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•  no  kinesthetic  cues  that  pilots  of  manned  aircraft  use  to  gain  an 
understanding  of  turbulence,  weather  conditions,  aircraft  movement  and 
gravitational  forces. 

Another  UAV  system  and  related  interface  that  the  author  had  a  chance  to  observe 
firsthand  was  the  “Bullseye”  interface  for  the  Raven  UAV  operated  by  WINTECS  during 
TNT  experimentation  at  Camp  Roberts  between  November  4  and  November  8,  2007  As 
was  mentioned  before,  a  relatively  simple  mishap  happened  during  TNT  experimentation 
that  yielded  some  clues  about  problematic  areas  with  the  interface  SA  relation.  The 
operator  had  limited  SA  due  to  the  necessity  to  fly  the  UAV  based  only  on  the  onboard 
camera  view  and  real  SA  is  gained  through  mission  controller’s  oral  directives  to  the 
operator.  Another  weak  point  for  the  pilot  was  the  lack  of  any  warning  about  low  altitude 
and  obstacles  on  the  route.  While  the  mission  controller  can  track  the  route  and  way 
points  from  the  map  within  the  interface,  if  mission  controller  becomes  engaged  with 
anything  other  than  the  mission,  the  operator  is  left  with  limited  SA. 

However,  when  the  two-man  team  can  work  together  without  distraction,  they  can 
execute  the  mission  with  no  problem.  A  feature  enabling  both  the  operator  and  the 
mission  controller  to  see  the  camera  field-of-view  (FOV)  increases  mission  effectiveness 
and  SA,  since  they  can  assess  the  video  feed  to  map  and  set  their  SA  easily  and  quickly 
by  recognizing  UAV  location.  This  helps  them  to  find  and  identify  terrain  features, 
waypoints,  and  eventually,  the  target. 
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Figure  30.  Screen  shot  from  Bullseye  SA  view 
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Figure  3 1 .  Camera  Field  of  View  in  Bullseye 


A  search  of  the  work  on  UAV  SA  views  yielded  different  approaches  to  SA  view 
designs.  One  interesting  design  tested  an  augmented  video  presentation  approach.87  In 
this  specific  workshop,  augmented  and  non-augmented  approaches  were  compared.  With 
the  augmented  video  display,  a  screen  displays  a  chase  aircraft  view  which  changes 


87  Jill  L.  Drury,  et  al.,  “Comparing  Situation  Awareness  for  Two  Unmanned  Aerial  Vehicle  Human 
Interface  Approaches,”  and  “Comparing  Situation  Awareness  for  Two  UAV  Interface  Approaches,” 
http://www.cerici.com/documents/20Q6  workshop/3- 1  -abst-MITREcorp-Drury.pdf  (accessed  15  January 
2008). 
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attitude  in  real  time  as  the  aircraft  flies  through  the  virtual  environment.  The  video  feed  is 
displayed  in  the  inset  box.  Researchers  geo-referenced  the  video  display  to  the  pre-loaded 
map  data,  which  appears  approximately  on  top  of  the  map  area  to  which  it  refers.  In  the 
non-augmented  video  display,  a  stationary  window  of  the  same  size  as  the  augmented 
display  is  shown.88  Researchers  claimed  that  with  the  augmented  approach,  “operators 
would  have  a  better  understanding  of  where  the  aircraft  was  with  respect  to  locations  on 
the  ground  using  Augmented  Virtuality  Interface  (AVI)  rather  than  unaugmented 
video.”89 


88  Drury,  et  al.,  “Comparing  Situation  Awareness  for  Two  Unmanned  Aerial  Vehicle  Human  Interface 
Approaches,”  and  “Comparing  Situation  Awareness  for  Two  UAV  Interface  Approaches.” 

89  Ibid. 
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Figure  33.  Unaugmented  Video  interface  display90 


Findings  from  the  experiments  favored  the  augmented  display  SA  view.  Results 
are  given  in  the  following  table.  Especially  worth  mentioning  is  difference  happens  at 
users  preference  of  AVI  interface  to  the  video  interface.  (As  2.2/7  to  5.8/7  preference  by 
the  users  would  indicate), 

As  is  apparent  from  the  results  in  Figure  34,  augmenting  the  video  feed  to  a  map 
gives  better  SA.  This  may  occur  because  it  feels  much  closer  to  the  feel  of  flying  the 
UAV  from  a  cockpit  inside  the  UAV,  thanks  to  the  forward  view  and  the  field  around  it 
being  presented  realistically.  Participants  in  the  research  project  pointed  this  out,  stating, 
“It’s  easier  to  recognize  where  the  UAV  is  relative  to  the  entire  search  space.”91 


90  Drury,  et  al.,  “Comparing  Situation  Awareness  for  Two  Unmanned  Aerial  Vehicle  Human  Interface 
Approaches,”  and  “Comparing  Situation  Awareness  for  Two  UAV  Interface  Approaches.” 

91  Ibid. 
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Result 

Video 

only 

AVI 

Significance 
of  difference* 

Average  error  between  marked 
and  truth  positions  for  victims 

66  mm 

54  mm 

p  <  .05 

Likert  scale  response  when  asked 
if  they  had  a  good  understanding 
of  the  UAV's  location  with  respect 
to  the  ground 

2.7/7 

3.9/7 

p  <  .003 

Likert  scale  response  when  asked 
if  interface  helped  to  perform  the 
search  task 

3.0/7 

3.9/7 

p  <  .003 

Likert  scale  response  when  asked 
for  interface  preference 

2.2/7 

5.8/7 

p  <.003 

Figure  34.  Augmented  video  design  test  results92 


C.  SA  VIEW  FOR  RASCAL  UAV 

Rascal  is  controlled  via  an  SA  view  named  “Situational  Awareness  (SA)  Multi 
Agent  System,”  which  was  created  within  NPS  specifically  for  this  project  by  Eugene 
Bourakov.  As  mentioned  in  overview  document  “Situational  Awareness  (Sa)  Multi  Agent 
System  Overview”  for  the  SA  view,  “SA  Multi  Agent  System  is  a  program  that  was 
developed  for  the  purpose  of  increasing  battlespace  awareness  of  ‘war  fighters’  and 
combatant  commanders  during  STAN  (Surveillance  and  Target  Acquisition  Network) 
and  TNT  (Tactical  Network  Topology)  experiments  that  are  run  each  quarter  at  NPS.”93 
Figures  35  and  36  show  the  current  interface  and  control  elements  on  the  SA  view. 


92  Drury,  et  al.,  “Comparing  Situation  Awareness  for  Two  Unmanned  Aerial  Vehicle  Human  Interface 
Approaches,”  and  “Comparing  Situation  Awareness  for  Two  UAV  Interface  Approaches.” 

93  Eugene  Bourakov,  “Situational  Awareness  (Sa)  Multi  Agent  System  Overview,” 
http://cenetix.nps.edu/cenetix/documents/SA  overview.pdf  (accessed  January  25,  2008). 
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Computer/UAS  Interface  and  Controls 


Designated 
Platform  to  be 
Controlled 


Figure  35.  Rascal  SA  view  interface  and  controls  I94 


94  Bourakov,  “Situational  Awareness  (Sa)  Multi  Agent  System  Overview.” 
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Computer/UAS  Interface  and  Controls  II 


Figure  36.  Rascal  SA  view  interface  and  controls  II95 


In  the  current  version  of  SA  view,  users  are  limited  to  available  maps  that  can  be 
chosen  from  the  list  within  the  view;  however,  the  list  of  available  maps  can  be  increased 
easily  to  include  maps  of  any  operational  area.  Entities  within  the  area  subject  to 
operation  can  be  represented  with  icons  on  the  map,  and  those  are  located  on  the  chart  in 
latitude-longitude  positions.  SA  agent  is  a  collaborative  tool  that  enables  users  to 
communicate  and  realize  dynamic  occurrences  in  the  area.  Users  can  send  text  messages 
to  each  other  using  the  message  box  tool  in  the  SA  agent  and  dragging  and  dropping  the 
mail  icon  on  the  recipient’s  icon. 

One  of  the  prime  features  of  the  view  is  that  it  enables  the  controller  of  the  UAV 
to  manipulate  the  actions  of  the  UAV  from  a  single  screen  with  relatively  simple 
commands.  The  operator  can  choose  between  a  video  camera  and  a  photo  camera  and  can 
further  arrange  camera  FOV  coverage.  The  ruler  feature  of  the  SA  view  gives  the 

9^  Bourakov,  “Situational  Awareness  (Sa)  Multi  Agent  System  Overview.” 
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capability  of  measuring  between  points  (Figure  37,  points  1,  2,  and  3);  in  addition,  it  can 
be  used  to  determine  the  coordinates  of  points  of  interest  (Figure  37,  point  4)  on  the  map 
(which  is  not  possible  with  another  way),  by  displaying  the  coordinates  in  the  designated 
box  at  the  bottom  left  comer  of  the  display  screen. 


Figure  37.  Ruler  function  in  use 


Figure  38  depicts  how  to  get  the  information  about  an  entity  on  the  screen.  After 
selecting  the  network  conductivity  icon  from  the  screen  (1),  dragging  and  dropping  this 
icon  to  top  of  any  entity  on  the  screen  (2)  results  in  a  pop-up  information  block  over  the 
screen  (3).  From  this  information  block,  the  “Info,”  “video,”  or  “network”  buttons  can  be 
selected  for  further  information  and  operations.  This  operation  is  depicted  in  sequence 
with  the  sequence  numbers  in  Figure  38. 
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Figure  38.  Information  acquirement  about  the  icons96 


Another  important  function,  sending  and  receiving  messages,  is  depicted  in  Figure 
39.  This  is  especially  an  important  function  for  the  controller  on  the  aircraft  because  it 
may  provide  the  only  communication  means  to  other  parties  within  the  network.  By 
clicking  the  sheet  icon  on  top  of  the  “text  message  window,”  the  user  opens  the  composer 
window.  After  writing  the  message,  the  user  drags  and  drops  the  envelope  icon  on  top  of 
the  entity  on  the  screen.  This  result  in  a  blinking  line  between  the  sender  and  the  receiver 
(on  the  sender’s  screen)  as  well  as  a  distinctive  incoming  message  sound  alert  on  the 
receiver’s  side.  This  operation  is  depicted  with  sequence  numbers  in  Figure  39. 


96  Landreth  and  Glass,  “Extending  the  Tactical  Horizon.” 
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Figure  39.  Communication  via  mail 
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Figure  40.  Rascal  SA  view  display  screen97 


97  Landreth  and  Glass,  “Extending  the  Tactical  Horizon.” 
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Tactical  Horizon  Extension  1 — Pelican 

15  Aug  2006 


Figure  4 1 .  Rascal  manned  aircraft  control  station98 

Within  all  the  functions  in  Rascal’s  capability,  the  most  important  is  probably  the 
“still  camera”  function.  As  discussed  in  a  recent  study,99  a  video  feed  seems  appealing  to 
pilots/operators,  allowing  them  to  navigate  by  a  continuous  image  feed  similar  to  the 
nonnal  visual  feed  to  the  brain.  But  from  a  quality  (and  quantity)  of  resolution  point  of 
view,  still  picture  cameras  complement  ISR  capabilities  better.  Moreover,  Commercial 
Off  The  Shelf  (COTS)  photo  cameras  are  more  cost  effective  than  video  cameras. 

In  the  case  of  Rascal,  implementation  is  done  by  “using  a  Canon  G7  digital 
camera  which  can  be  operated  wirelessly  from  an  operator  on  the  ground,  providing 
focus,  zoom,  capture  and  download  capabilities,  all  while  the  UAS  is  in  flight.”100 
Installation  of  the  camera  to  Rascal  UAV  is  shown  in  Figure  42. 

98  Landreth  and  Glass,  “Extending  the  Tactical  Horizon.” 

99  Kevin  Jones,  et  al.,  “High  Resolution  Imaging  from  a  Small  Unmanned  Air  Vehicle,”  (November 
2007)  http://ieeexplore.ieee.Org/iel5/l  1005/34689/0 16564 18.pdf  (accessed  February  16,  2008). 

100  Ibid. 
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Figure  42.  Installation  of  Canon  G7  camera  to  Rascal  UAV101 


Without  going  into  all  the  technical  details,  in  the  current  version,  many  camera 
functions  are  controlled  wirelessly  via  USB.  Then  pictures  taken  are  stored  on  the  remote 
computer  hard-drive  in  the  UAV  rather  than  in  the  camera’s  memory.  Image  metadata 
enriched  by  UAV  metadata  is  stored  with  the  picture.  An  example  of  metadata  stored  in  a 
typical  transmission  package  is  given  in  Figure  43.  As  can  be  seen  from  the  figure,  the 
camera  generates  the  first  25  lines  of  the  metadata,  and  the  CCM  software  adds  the  rest; 
these  are  related  to  aircraft  metadata  like  altitude,  speed,  coordinates,  and  so  forth  at  the 
time  the  shutter-release  command  is  sent.102 


101  Jones,  et  at.,  “High  Resolution  Imaging  from  a  Small  Unmanned  Air  Vehicle.” 

102  Ibid. 
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Figure  43.  Example  of  Metadata  stored  in  a  typical  image 


Considering  the  limited  bandwidth  between  the  UAV  and  the  other  station(s), 
instead  of  sending  the  entire  high-resolution  picture  (which  would  take  minute(s)),  a 
thumbnail  image  is  downloaded  and  displayed  in  the  SA  view.  Checking  the  thumbnail, 
the  user  can  decide  to  download  the  full-resolution  image  or  not,  eliminating  unnecessary 
process  and  bandwidth  use.103  Graphic  User  Interface  (GUI)  used  for  a  G7  camera  and 
the  thumbnail  images  downloaded  from  the  Rascal  UAV  can  be  seen  in  Figure  44. 
Clicking  on  those  thumbnail  images  result  in  a  download  of  the  full-resolution  images. 

I®3  Jones,  et  at.,  “High  Resolution  Imaging  from  a  Small  Unmanned  Air  Vehicle.” 
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Figure  44.  GUI  for  controlling  G7  and  thumbnail  images104 

With  the  additional  improvements  on  the  software,  the  capability  to  automatically 
arrange  the  timing  of  the  photography  in  the  desired  flight  path  is  gained.  Furthermore, 
the  images  taken  may  be  geo-rectified  in  an  overlay  with  Google  Earth.  A  sample  image 
taken  during  TNT  07-02  is  given  in  Figure  46  as  overlaid  on  a  Google  Earth  image;  the 
raw  image  from  Google  Earth  is  shown  in  Figure  45. 


104  Jones,  et  at.,  “High  Resolution  Imaging  from  a  Small  Unmanned  Air  Vehicle.” 
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Figure  45.  Sample  raw  image  from  Google  Earth 
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Figure  46.  Image  overlaid  in  Google  Earth 

D.  IDENTIFIED  AREAS  OF  PROBLEMS  AND  REQUIREMENTS  FOR  THE 
SA  INTERFACE 

Interfaces  that  were  examined  in  this  study  raised  a  variety  of  problems  and 
relative  solutions.  However,  those  problems  and  solutions  should  be  evaluated  in  their 
relative  environments  and  with  regard  to  their  mission  concepts. 

The  defined  requirements  set  for  aviation  display  design  of  Wickens  and 
Hollands105  will  be  used  to  measure  the  convenience  of  the  user  interface,  and  then  to  try 
to  address  some  of  the  shortcomings  of  the  SA  view.  Wickens  and  Hollands  described 
seven  critical  principles  of  aviation  display  design,  which  are: 


105  Robyn  Hopcroft,  Eleanore  Burchat  and  Julian  Vince,  “Unmanned  Aerial  Vehicles  for  Maritime 
Patrol,”  http://stinet.dtic. mil/cgi-bin/GetTRDoc?AD=ADA454918&Location=U2&doc=GetTRDoc. pdf 

(accessed  January  24,  2008). 
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•  Principle  of  Information  Need.  Information  that  is  required  more 

frequently  should  be  displayed  in  the  most  accessible  locations; 

•  Principle  of  legibility.  Displays  must  be  legible  to  be  useful:  That  is,  large 
enough  and  with  adequate  contrast,  brightness,  illumination,  volume  (for 
auditory  displays),  etc.; 

•  Principle  of  display  integration/proximity  compatibility  principle. 
Information  sources  that  need  to  be  integrated  or  compared  should  be 
positioned  in  close  physical  proximity  on  the  display; 

•  Principle  of  pictorial  realism.  The  display  should  be  a  pictorial 

representation  of  the  infonnation  that  it  represents; 

•  Principle  of  the  moving  part.  The  moving  element  on  a  display  should 
correspond  to  the  moving  element  in  the  operator’s  mental  model; 

•  Principle  of  predictive  aiding.  Predictive  infonnation  regarding  future 
aircraft  state  is  valuable  (as  long  as  it  is  accurate  and  easily  understood), 
and; 

•  Principle  of  discriminability.  A  display  element  in  a  certain  context 
should  never  look  (or  sound)  similar  to  another  element  that  could  occur  in 
the  same  display  context.106 

However,  those  principles  are  defined  for  manned  aviation,  especially  for  the 
aircraft  cockpit.  Even  though  a  UAV  interface  should  relate  to  cockpit  design  because 
both  are  interfaces  for  the  operator  of  the  air  vehicle,  a  UAV  interface  has  its  own 
challenges  and  all  the  display  items  for  a  manned  aircraft  operator  (pilot)  may  not 
necessarily  be  needed  for  the  UAV  operator.  However,  those  principles  may  still  be 
referred  to  for  determining  the  necessities  for  an  interface  for  the  purposes  of  this  thesis. 

Another  specialty  in  the  UAV  case  is  the  level  of  automation.  According  to  TNT 
faculty  member  Vladimir  Dobrokhodov,  UAVs  are  not  really  piloted;  in  a  more  correct 
sense,  the  controllers  augment  the  autopilot.107  This  means  that  every  command  the  pilot 
gives  to  the  air  vehicle  is  in  fact  a  command  to  the  autopilot  to  command  the  air  vehicle. 
All  the  unique  characteristics  of  unmanned  aviation  define  a  unique  set  of  necessities  or 
requirements  for  the  subject  of  this  thesis. 


106  Hopcroft,  Burchat,  and  Vince,  “Unmanned  Aerial  Vehicles  for  Maritime  Patrol.” 

107  Vladimir  Dobrokhodov,  personal  communication. 
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The  SA  view  that  is  the  subject  of  this  thesis  is  the  SA  view  used  for  the  NPS 
Rascal  Unmanned  Aerial  System  (UAS),  and,  more  specifically,  that  used  while  flying 
the  Rascal  from  an  aircraft,  which  is  the  Pelican  in  this  case.  It  should  be  remembered 
that  the  thesis  done  by  Landreth  and  Glass  is  a  work  of  concept,  and  thus  ideal  conditions 
should  not  be  expected.  So,  even  though  it  may  be  used  with  a  more  stabilized  and  unique 
set  of  conditions  in  the  future,  at  the  moment,  this  operation  is  executed  via  mobile 
platforms,  namely  laptops.  That  being  said,  the  same  concept  can  be  used  by  the  units  in 
an  operation  area  with  available  flying  platforms  assigned  to  military  units.  As  is 
displayed  in  Figure  47,  the  conditions  of  the  controller  for  the  specific  type  of  the  UAV 
make  each  case  unique. 

The  controller  is  limited  to  one  laptop  screen,  has  limited  space,  and  executes  the 
mission  in  flying  conditions.  What  this  means  is  that,  during  the  mission,  the  controller  is 
subject  to  the  effects  of  aircraft  maneuvers,  different  light  and  temperature  conditions 
throughout  the  flight  route,  turbulence,  vibrations,  and  so  forth.  So  some  of  the  findings 
from  different  experiments  for  interface  designs  may  not  be  applicable  to  these  purposes. 
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Figure  47.  Controlling  Rascal  from  Pelican108 


Some  of  the  shortcomings  or  problem  areas  that  were  identified  in  conjunction 
with  Wickens  and  Hollands’  principles  are  as  follows. 


1.  Problems  regarding  the  SA  of  the  Operator 

Even  though  the  Rascal  SA  view  is  not  as  crippled  as  the  Raven  SA  view  for  the 
pilot  (in  the  sense  of  SA  deficiencies  from  the  operator/pilot  point  of  view)  and  has  its 
pros  (such  as  that  the  video  feed  display  can  be  moved  to  anywhere  on  the  screen,  and 
resized  to  provide  a  more  thorough  view  or  occupy  less  space),  it  still  lacks  the  complete 
SA  gain  for  the  pilot  of  the  UAV  in  terms  of  video  feed  representation  and  operator  SA. 
When  video  camera  feed  is  displayed  in  the  screen,  it  comes  with  a  smaller  display  screen 
on  the  main  display  screen,  independent  from  the  area  on  the  map  that  the  camera  is 
shooting.  That  is,  the  video  screen  location  is  not  necessarily  displayed  on  top  of  the 

108  Landreth  and  Glass,  “Extending  the  Tactical  Horizon.” 
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actual  field  of  interest  that  the  video  camera  is  shooting.  This  situation  conflicts  with  the 
“ Principle  of  display  integration/proximity  compatibility  principle,  ”  because  the  source 
and  the  display  are  not  positioned  in  close  or  related  proximity  and  the  “ Principle  of 
pictorial  realism ”  to  some  degree,  because  the  display  is  not  realistically  displayed  in  this 
case. 


Figure  48.  Video  feed  display  on  Rascal  SA  view109 

There  is  nothing  in  the  display  to  rebuild  the  pilot’s  orientation  in  the  case  of 
orientation  loss,  meaning  the  loss  of  the  feeling  and  knowledge  of  up-down,  south-north, 
close-away  from  a  point,  etc.  When  the  operator  gets  busy  with  something  in  hand, 
he/she  may  get  too  focused  on  that  thing,  so  that,  when  the  time  comes  to  continue  the 
mission  for  another  point  or  location,  he  may  have  lost  his  knowledge  of  where  the  UAV 
is  in  regard  to  other  entities  in  the  area  or  to  the  area  itself.  Due  to  such  loss  of 


Dobrokhodov,  et  at,  “New  Generation  of  Rapid  Flight  Test  Prototyping  System.” 
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orientation,  (which  also  may  be  identified  as  loss  of  SA,  especially  perception,  in  tenns 
of  answering  the  “where  am  I?”  question),  the  operator  may  lose  valuable  time  in 
operating  conditions. 

Another  deficiency,  which  was  identified  regarding  the  SA  of  the  operator,  is 
related  to  the  perception  term  of  Endley’s  SA  definition.  Current  SA  does  not  enable  an 
operator  to  identify  his  position  in  compliance  with  the  other  entities  within  the  area,  or 
vice  versa.  At  any  given  time,  an  operator  cannot  determine  distance,  heading  or  time  to 
any  entity  or  point  within  the  area.  The  only  tool  that  can  be  used  in  this  condition  is  the 
ruler  located  on  the  right  side  of  the  screen.  As  pointed  out  before  when  the  Rascal  SA 
view  was  introduced,  an  operator  can  use  the  ruler  to  measure  the  distance  from  a  point  to 
the  UAV’s  location  at  the  time  or  to  any  other  point.  In  Figure  49,  this  function  is 
displayed.  The  green  line  between  “alert  4”  and  the  point  of  focus  of  the  photo  camera 
shows  the  line  drawn  by  the  ruler,  and  the  value  of  “1.882  km”  shows  the  distance 
measured  between  two  points. 
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Figure  49.  Ruler  function  usage  in  current  version 


A  final  deficiency  that  was  identified  regarding  problems  related  to  the  SA  of  the 
operator  is  the  lack  of  any  reference  mechanism  for  coordination  between  assets  to 
identify  a  point.  This  problem  was  observed  during  TNT  experiments  between  November 
4  and  November  8,  2007,  at  Camp  Roberts  CA.  While  one  of  the  stations  using  an  SA 
view  was  trying  to  describe  the  location  of  a  suspected  vehicle  to  the  ground  station  on 
the  van,  precious  time  was  lost  because  of  a  slight  difference  between  the  describing 
party’s  intended  point  and  the  description  understood  by  the  receiving  point  party.  Due  to 
this  misunderstanding,  the  UAV  flew  over  the  wrong  location  repeatedly  in  an  effort  to 
identify  the  suspected  vehicle. 
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2. 


Problems  due  to  the  Nature  of  Flight  Conditions 


Another  problem  from  the  aspect  of  the  operator  in  the  aircraft  is  that  the  operator 
is  subject  to  the  conditions  in  an  aircraft,  including  the  light  coming  from  ever-changing 
angles  during  flight,  throughout  the  mission.  This  may  cause  the  operator  to  lose  visual 
sight  of  the  laptop  screen,  and  halt  operation.  One  moment,  he/she  can  have  a  great 
vision,  and  seconds  later,  might  lose  it  completely  with  a  turn.  There  are  also  the  night 
flight  conditions;  light  coming  out  from  laptop  screen  can  annoy  the  pilot  of  the  manned 
aircraft.  This  situation  creates  a  confliction  with  the  “Principle  of  legibility.”  Yet  another 
problem  due  to  the  same  reason  is  the  possibility  of  giving  erratic  commands  or 
commands  that  are  not  finessed  over  the  laptop  due  to  all  the  vibration  and  rough/bumpy 
flight  conditions  that  might  be  encountered  during  the  flight. 

3.  Problems  due  to  Lack  of  Alerts  to  the  Operator 

The  last  class  of  problems  identified  is  the  lack  of  alerts  to  the  operator.  At  the 
current  state  of  operation,  the  operator  in  the  aircraft  has  no  means  of  getting  an 
indication  of  possible  problems.  The  only  time  he/she  is  going  to  learn  about  a  problem  is 
when  it  happens,  be  it  engine  malfunction,  connection  loss  with  other  stations  or  the 
UAV,  closure  to  restricted  zone,  etc.  In  addition,  any  potentially  dangerous  situation  like 
low  altitude  flying  can  go  unnoticed  due  to  the  lack  of  an  alert  mechanism.  In  conjunction 
with  the  flight  conditions,  incoming  messages  may  remain  unnoticed,  since  the  operator 
may  have  a  hard  time  hearing  the  email  received  sound  alert.  These  problems  are  in 
conflict  with  the  principles  of  “ Principle  of  display  integration/proximity  compatibility 
principle’’  and  “Principle  of  information  need.  ’’ 
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IV.  FINDINGS  AND  SUGGESTIONS  FOR  AN  AIRCRAFT  PILOT 
SITUATIONAL  AWARENESS  INTERFACE  FOR  AIRBORNE 

OPERATION 


Based  on  this  researcher’s  experience  as  an  operational  pilot,  and  observations 
from  the  review  of  the  related  literature,  some  suggestions  for  the  Rascal  interface  were 
developed. 

A.  POSSIBLE  SOLUTIONS  FOR  PROBLEMS  FOUND 

While  searching  for  solutions  to  identified  problem  areas,  the  special  situations 
related  to  the  Rascal  UAV,  its  mission  conditions,  and  its  SA  view  were  considered.  The 
current  view  offers  flexibility,  low  cost  and  ease  of  implementation,  all  of  which  should 
be  retained.  Using  a  laptop  (or  even  a  smaller  medium  like  a  PDA)  for  the  control  of  the 
UAV  enables  the  operators  to  operate  from  different  kinds  of  platforms,  since  they  are 
not  tied  completely  into  a  specific  platfonn.  Furthermore,  there  is,  in  reality,  only  so 
much  bandwidth  and  computing  power  available  at  a  reasonable  cost. 

1.  Problems  regarding  the  SA  of  the  Operator 

While  examining  the  current  literature  about  SA  displays,  one  SA  view  design 
stood  out;  it  is  depicted  in  Figure  50.  In  this  view,  the  UAV  is  depicted  from  the  chase 
aircraft  view  and  the  video  feed  is  embedded  on  top  of  the  display  map  at  the  actual 
location  the  video  is  shooting.  This  technique  mitigates  the  “looking  through  soda  straw” 
feeling,  because  the  operator  of  the  UAV  actually  can  sense  the  area  other  than  where  the 
video  is  shooting,  and  can  have  better  SA  since  he/she  can  evaluate  the  UAV’s  location 
and  the  environment  surrounding  the  UAV.  Also,  the  operator  does  not  need  to  move 
his/her  eyes  through  different  locations  over  the  display  to  create  a  meaningful  picture  of 
the  area  in  the  mind  and  match  the  video  feed  to  the  area.  One  other  capability  gained 
with  the  design  is  a  chase  aircraft  view.  A  chase  aircraft  view  creates  a  more  inside  the 
UAV  feel  than  the  current  view  of  the  UAV  from  sky  (or  god’s  eye/bird’s  eye  view). 
However,  this  display  comes  with  its  own  challenges.  There  is  always  added  computation 
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power  need  with  such  a  design  and  the  laptop  is  limited  in  this  regard  at  the  moment.  In 
addition,  there  is  a  bandwidth  problem  and  this  kind  of  design  would  require  additional 
bandwidth  to  keep  the  steady  video  feed  to  stations  from  the  UAV.  Another  limitation 
due  to  use  of  a  laptop  is  the  fact  of  a  limited  screen.  Depending  on  the  altitude  at  which 
the  mission  is  executed,  the  video  feed  can  be  so  small  on  the  map  that  identifying  the 
objects  on  the  video  feed  can  be  really  hard,  or  it  may  be  so  big  that  it  may  cover  a  big 
part  of  the  screen  to  the  point  that  the  entire  gained  SA  might  be  lost.  Even  the  focus 
feature  of  the  camera  may  not  be  helpful.  Also,  a  chase  aircraft  view  requires  that  a  view 
of  the  map  be  angled  to  match  the  video  feed  and  chase  view  in  order  to  be  realistic. 

Based  on  these  considerations,  the  resizable  and  not  embedded  video  feed  screen 
display  of  the  current  design  for  the  Rascal  UAV  is  favored.  At  this  point,  it  is  important 
to  remember  that  the  maps  of  the  areas  are  preloaded  on  the  SA  view,  and  embedding  the 
video  feed  as  in  Figure  50  can  be  a  technical  challenge.  An  attempt  was  made  to  come  up 
with  suggestions  that  offered  minimal  added  complexity  over  the  current  design. 


Figure  50.  Embedded  video  feed  to  display  screen110 


110  Drury,  et  at,  “Comparing  Situation  Awareness  for  Two  Unmanned  Aerial  Vehicle  Human 
Interface  Approaches,”  and  “Comparing  Situation  Awareness  for  Two  UAV  Interface  Approaches.” 
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To  address  these  points,  another  design  example  encountered  while  researching 
this  project  was  utilized:  the  “Bullseye”  SA  view  of  the  Raven  UAV  operated  by  the 
WINTEC.  The  appealing  feature  of  the  Bullseye  design  is  a  rather  simple  addition  that 
creates  added  SA  according  to  mission  controller  and  operator  involved  in  TNT 
experiments  at  Camp  Roberts  on  November  6,  2007.  This  simple  addition  is  a  trapezoid 
representing  the  camera  FOV  coverage  on  the  map,  shown  inside  the  red  circle  in  Figure 
51.  Although  one  may  think  it  is  already  apparent  where  the  camera  is  shooting,  in 
reality,  knowing  the  approximate  borders  of  the  camera  coverage  makes  matching  the 
video  feed  to  the  area  easier. 


Figure  5 1 .  Bullseye  SA  view  for  Raven  UAV  by  WINTEC 


75 


The  suggested  design  for  this  problem  is  depicted  in  Figure  52.  With  the  basic 
addition  of  the  pictural  display  of  camera  FOV  coverage  on  the  map,  the  area-video  feed 
match  is  improved.  This  feature  is  already  present  with  the  current  SA  view  for  the  photo 
camera  view  and  can  be  arrasnged  according  to  altitude  and  zoom  level  as  shown  in 
Figures  53  and  54. 


Figure  52.  Suggested  SA  view  VI  (version  1) 
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Figure  53.  Before  altitude  and  FOV  degree  change 
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Figure  54.  After  altitude  and  FOV  degree  change 


A  simple  north  arrow  or  a  compass  rose  on  the  map  can  help  the  operator  in 
orienting  himself/herself  to  area.  Also,  giving  the  heading  and  distance  to  a  point  or  entity 
from  the  UAV  by  clicking  on  the  entity  would  enhance  this  orientation. 

As  was  discussed  earlier,  in  the  current  SA  view,  maps  are  preloaded  and  fixed 
for  the  area  of  interests.  As  a  result,  there  are  no  navigation  options  like  ‘north  up” 
(which  orients  the  map  to  show  north  on  the  upper  side  all  the  time)  or  “route  up”  (which 
orients  the  map  to  show  the  route  direction  all  the  time).  The  proposed  modification  to 
the  SA  view  should  help  overcome  this  issue.  Since  the  area  maps  are  fixed  and 
preloaded  for  the  mission  area,  adding  a  symbol  pointing  to  north  is  easier  and  does  not 
have  to  change  over  the  map,  because  the  user  can’t  change  the  area  map  orientation  on 
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the  screen.  So  a  basic  north  arrow  or  compass  rose  should  be  added  to  the  top  right  part 
of  the  screen;  these  can  be  relocated  in  order  not  to  hinder  the  operations  by  blocking  the 
screen. 


Figure  55.  Suggested  view  with  north  arrow  or  compass  rose 


When  it  comes  to  displaying  distance  and  heading  to  an  entity,  two  different 
approaches  were  considered:  first,  adding  a  feature  that  would  display  distance  and 
heading  to  any  point  or  entity,  when  this  entity  or  point  is  clicked;  and  second, 
embedding  a  compass  rose  over  the  UAV  icon  (or  any  entity  in  this  matter)  and  using  the 
ruler  function  to  read  the  distance  while  reading  the  heading  manually  over  the  compass 
rose.  However  the  second  version  would  not  bring  a  significant  new  capability  to  the 
table,  since  the  operator  still  needs  to  use  the  ruler  to  get  the  distance.  Given  the  current 
flight  conditions  in  the  Pelican,  it  is  preferable  to  minimize  manual  commands. 
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Figure  56.  Suggested  view  with  distance  heading  function  version  1 
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Figure  57.  Suggested  view  with  distance  heading  function  version  2 


Finally,  for  the  problem  of  a  lack  of  a  common  reference  mechanism  between 
participating  parties,  the  simple  yet  rather  effective  mechanism  of  implementing  a  grid 
mechanism  over  the  area  map  is  suggested. 

With  the  current  view,  creativity  in  using  the  existing  features  makes  a  partial 
solution  possible.  This  solution  can  be  possible  by  using  “Alert”  icons  on  the  left  side  of 
the  screen.  The  party  describing  a  point  can  locate  one  of  the  (predetermined)  alert  icons 
on  the  intended  point  on  the  map,  and  the  other  parties  can  then  identify  the  location. 
However,  if  the  alert  icons  are  already  in  use,  adding  another  alert  icon  on  the  map  may 
cause  confusion  between  participating  parties.  One  of  the  parties  may  confuse  one  of  the 
already-existing  icons  for  the  newly-added  icon  identifying  the  marked  location.  (Even 
though  icons  are  in  different  colors,  they  have  the  same  shapes  and  patterns). 
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Due  to  the  possibility  of  such  confusion,  another  solution  is  suggested:  Adding  a 
grid  view  over  the  map.  This  grid  view  would  activate  when  the  “grid”  button  is  selected 
(causing  the  button  to  turn  green),  and  deactivate  when  the  button  is  reselected  (causing 
the  button  to  turn  red).  A  simple  depiction  of  this  suggestion  is  given  in  Figure  58,  below. 


Situational  Awareness 

u  am  cwrn.  wv  no)  xn 


Figure  58.  SA  view  with  grid  imposed  over  the  map 


2.  Problems  due  to  the  Nature  of  Flight  Conditions 

There  are  also  problems  that  arise  from  the  fact  that  the  controller  is  located  on  an 
aircraft.  Again  by  staying  within  the  limits  described  earlier,  some  simple  but  probably 
effective  solutions  for  those  kinds  of  problems  are  suggested.  The  main  problem  that  was 
identified  was  that  light  coming  from  different  directions  throughout  the  mission  can 
reflect  at  different  angles  and  can  make  the  screen  unreadable  at  times.  A  simple  foldable 
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hood  approach  can  solve  this  problem  for  the  most  part  while  staying  within  this  project’s 
limits.  This  approach  was  also  used  by  the  WINTEC  team  for  the  operator  of  the  Raven 
UAV. 

Yet  another  problem  identified  under  this  category  was  difficulty  in  putting  the 
commands  on  the  laptop  finger  pad  while  experiencing  vibrations,  aircraft  maneuvers, 
and  turbulence.  A  mouse  pad  and  wireless  mouse  were  considered  in  an  attempt  to 
mitigate  this  problem,  but  they  also  had  downsides  like  lack  of  space  and  additional  gear 
required  to  keep  them  under  control.  Eugene  Bourakov  suggested  a  better  idea:  a  finger 
roll  type  mouse. 

This  mouse,  suggested  for  use  in  an  airborne  control  platform  due  to  limited  space 
in  Pelican  cockpit,  is  shown  in  Figure  59.  The  scroll  ball  (track  ball)  on  the  mouse 
enables  the  operator  to  move  a  pointer  in  the  screen,  while  two  buttons  on  top  serve  as  the 
“right  and  left  click  buttons”  of  the  standard,  widely-used  mouse  design.  The  trigger  type 
button  in  the  bottom  enables  the  operator  to  use  the  right  click  function  of  the  standard 
mouse  while  moving  the  pointer  on  the  screen  with  the  scroll  ball.  This  mouse  design 
helps  the  operator  in  the  aircraft  to  give  commands,  send  messages,  etc.,  without  the 
limits  (such  as  the  need  for  a  flat  surface,  etc.)  of  the  standard  mouse  design. 
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Figure  59.  Finger  roll  type  mouse 


3.  Problems  due  to  Lack  of  Alerts  to  the  Operator 

As  previously  mentioned,  there  is  no  alert  mechanism  implied  on  the  SA  view, 
and  any  potential  problems  may  go  unnoticed  until  they  actually  happen. 

In  the  current  situation,  the  ground  station  gets  the  information  regarding  engine 
parameters,  communication  links,  and  the  health  of  the  UAV.  But  those  are  not 
transferred  to  the  operator  on  the  aircraft.  Also,  some  other  important  parameters,  like 
low  altitude  or  slow  speed,  which  may  indicate  upcoming  dangers,  go  unnoticed  because 
there  are  no  means  to  engage  operator’s  attention. 

One  other  important  point  that  may  go  unnoticed  is  incoming  messages.  As 
mentioned  earlier,  messages  can  become  the  only  means  of  communication  between 
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stations  and  may  be  vital  to  mission  success.  In  the  current  version,  a  new  message 
creates  a  sound  alert  on  the  receiver’s  screen.  However,  this  alert  may  go  unnoticed  due 
to  the  noise  level  in  the  aircraft  and  the  workload  of  the  operator. 

Most  of  the  UAV  operations  including  Rascal,  are  largely  automated  flights; 
when  the  operator  intervenes  to  control  the  UAV,  his  commands  are  actually  commands 
to  the  autopilot  to  command  the  UAV.  Automation  in  aviation  can  lead  to  some  problems 
like  a  requirement  for  increased  monitoring/watch  keeping,  failure  to  monitor  progress  of 
the  flight,  and  minor  input  errors  with  serious  consequences.111  An  alert  mechanism 
plays  the  role  of  circuit  breaker  in  these  conditions,  by  letting  the  operator  know  when 
things  are  going  wrong  that  might  go  unnoticed  otherwise. 

Our  suggestion  to  deal  with  the  lack  of  an  alert  mechanism  was  to  create  a  caution 
mechanism  on  the  SA  view  similar  to  the  ones  in  aircraft.  A  “Master  Caution”  alert  on 
the  screen  is  suggested  to  visually  catch  the  attention  of  the  operator.  This  “Master 
Caution”  alert  should  appear  in  the  middle  of  the  screen  in  a  red  blinking  box.  At  the 
same  time,  this  box  should  be  transparent  to  some  level,  in  order  not  to  halt  the 
continuing  mission  while  serving  its  purpose.  In  compliance  with  the  “Master  Caution” 
alert  in  the  middle  of  the  screen,  there  should  also  be  a  Master  Caution  panel,  with  a 
blinking  red  box  for  the  respective  malfunction.  Any  malfunction  or  dangerous  situation 
should  activate  a  “Master  Caution”  alert  and  respective  “Caution”  light.  This  design 
suggestion  is  depicted  in  Figure  60.  Thresholds  for  those  respective  alerts  should  be  set 
either  permanently  or  per  mission  (engine  speed,  altitude,  etc.).  How  those  alerts  will  be 
evoked  is  the  subject  of  another  discussion. 

For  incoming  messages,  a  similar  approach  with  slight  variations  is 
recommended.  For  incoming  messages,  a  yellow  blinking  “New  Message”  ellipse  should 
appear  at  the  top  middle  of  the  screen.  This  suggested  design  is  also  depicted  in  Figure 
60.  These  variations  in  location  shape  and  color  would  prevent  confusion  between  the 
two  master  alerts.  This  measure  also  complies  with  the  “ Principle  of  discriminability.  ” 

1 1 1  Anthony  Ciavarelli,  “Cockpit  Control  and  Display  Design  Hazard  Analysis,” 
https://www.cnet.navy.mil/nascweb/sas/files/cockpit  control  and  display  hazard  checklist.pdf  (accessed 

March  5,  2008). 


85 


Figure  60.  Recommended  SA  view  design  before  meeting 


After  candidate  additions  to  SA  view  for  Rascal  were  developed,  Dr.  Bourakov 
was  consulted  for  further  refinement  of  the  design.  Dr.  Bourakov  suggested  that  the 
mission  screen  should  be  kept  less  crowded  with  additional  icons  and  alerts,  since  the 
screen  has  limited  space  and  blocking  this  available  space  might  hinder  the  operation. 
Instead  of  putting  alerts  on  the  main  screen,  he  recommended  putting  them  on  the  left 
side  of  the  screen.  A  blinking  message  box  when  a  new  message  is  received  would  take 
the  place  of  a  “new  message”  icon  in  the  main  screen.  For  the  “master  caution”  box,  he 
recommended  that  the  blinking  master  caution  box  be  put  on  top  of  the  message  box, 
which,  when  clicked,  would  activate  an  additional  master  caution  panel  with  the  related 
information  about  the  cause  of  the  “master  caution.”  These  additions  would  comply  with 
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Dr.  Chiavarelli’s  notion  of  warning  signals  being  adequate  to  gain  the  attention  of  the 
crew112  while  invading  less  space  in  the  limited  screen. 

Applying  his  suggestions  to  the  next  version  of  the  SA  view  resulted  in  the  SA 
screen  depicted  in  Figure  61.  According  to  this  screen,  any  condition  triggering  a  Master 
Caution  light  blinking  action  would  lead  the  operator  to  click  on  the  master  caution  (1) 
icon.  Clicking  on  the  master  caution  light  would  cause  a  “Warning  Panel”  to  come  up  on 
the  screen  (2).  On  this  panel,  the  conditions  that  activated  the  master  caution  would  be 
displayed  with  different  color  codes  depending  on  their  urgency  (more  important  ones  in 
red,  others  in  yellow).  If  the  operator  wants  to  get  more  detailed  infonnation,  clicking  the 
“Details”  button  next  to  the  threat  would  result  in  a  “Warning  Detail  Panel”  opening  up 
on  the  screen.  This  details  can  give  the  time  of  detection  of  the  threat,  current  state,  worst 
condition,  and  the  related  times.  It  is  important  that  alert  infonnation  be  prioritized,  since, 
even  in  a  condition  where  all  the  necessary  information  is  depicted,  operator(s)  can 
overlook  vital  information  while  focusing  on  less  important  information.  As  research 
shows,  35%  of  SA  errors  involved  situations  where  all  the  needed  information  was 
present,  but  was  not  attended  to  by  the  operator.113 

All  of  the  panels  that  have  opened  on  the  screen  can  be  relocated  by  clicking  and 
dragging  over  the  screen,  and  can  be  resized  so  as  not  to  hinder  ongoing  operations. 


112  Ciavarelli,  “Cockpit  Control  and  Display  Design  Hazard  Analysis.” 

113  Mica  R.  Endsley  and  Daniel  J.  Garland,  Situation  Awareness  Analysis  and  Measurement,  (New 
York:  Lawrence  Erlbaum  Associates,  2000). 
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Figure  6 1 .  Recommended  S A  view  design  Version  2  (V2) 
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V.  SAMPLE  INTERFACE  DESIGN 


A.  PROPOSED  FEATURES  FOR  THE  SA  VIEW 


This  chapter  summarizes  the  proposed  SA  view  design  with  all  the  new  additions 
displayed.  The  SA  view  before  the  proposed  changes  are  presented  one  more  time  to 
make  the  comparison  easier  for  the  reader.  All  the  proposed  solutions  are  intended  to  be 
cheap  and  easy  to  implement.  The  proposed  changes  are  also  designed  to  preserve  the 
flexibility  of  the  current  SA  interface,  since  this  feature  is  unique  and  important  for  user 
preference,  ease  of  use,  and  the  eventual  success  of  the  SA  view. 


Figure  62  depicts  the  current  SA  view.  Figure  63  displays  the  proposed  additions 
to  the  current  SA  view:  (1)  Master  Caution  panel,  (2)  Flashing  Message  box  in  case  of 
message  received,  (3)  Camera  Field  of  View  Indicator,  (4)  Grid  Button,  (5)  Distance 
Heading  Button,  and  (6)  North  Arrow. 


Figure  63.  SA  interface  with  proposed  features  added 


The  following  figures  display  the  features  added  in  use.  Figure  64  displays  the 
Master  Caution  panel.  This  feature  fills  an  important  hole  in  the  current  design  from  the 
point  of  the  UAV  operator  in  the  manned  aircraft.  These  cautions  will  enable  the  operator 
to  take  necessary  steps  instead  of  watching  helplessly  when  emergencies  happen.  The 
types  of  cautions  and  thresholds  should  be  flexible  in  order  to  be  able  to  change 
according  to  mission  and  UAV  types  and  requirements.  This  work  does  not  suggest 
certain  types  and  thresholds.  However,  at  a  minimum,  low  RPM,  low  power  and  low 
altitude  cautions,  as  well  as  link  health  cautions  should  be  included. 
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Figure  64.  Master  Caution  Feature 


Another  feature  is  displayed  in  Figure  65.  Grid  implementation  is  a  useful 
reference  tool  for  coordination  of  the  efforts  of  the  parties  participating  in  the  mission. 
This  feature  should  minimize  confusion  between  the  parties  in  their  efforts  to  identify 
target  locations,  as  well  as  defining  sectors  or  areas  on  the  map. 
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F igure  6  5 .  Grid  F eature 


Another  proposed  feature  is  a  bearing/heading  feature  and  north  arrow  on  the 
map.  Both  features  will  minimize  the  time  for  the  pilot  to  orient  himself/herself  on  the 
field  while  navigating  between  points  or  searching  for  targets.  They  will  also  enable  the 
operator  to  understand  his  position  in  relation  to  other  entities  in  the  field. 
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Figure  66.  Distance  Heading  Feature  and  North  Arrow  Feature 


B.  DESIGN  TESTING  AND  EVALUATION 

When  this  thesis  was  begun,  the  plan  was  to  identify  the  weaknesses  and  missing 
features  in  the  current  SA  interface,  find  the  appropriate  solutions  to  those  weaknesses 
and  propose  further  solutions  for  the  missing  ones.  After  developing  applicable  solutions, 
the  plan  was  to  implement  those  modifications  to  the  SA  view,  and  then  test  them  in  the 
field  during  the  TNT  experiment. 

However,  both  a  relatively  short  timeline,  and  the  busy  schedules  of  the  thesis 
advisors  limited  the  implemented  changes.  Furthermore,  during  the  TNT  experiment 
from  February  25  to  29,  it  was  learned  that  the  SA  view  usage  for  the  control  of  the 
Rascal  UAV  had  changed  radically.  The  initial  plan  was  to  use  the  SA  view  from  the 
Pelican  manned  aircraft  during  the  TNT  experiment.  But  the  configuration  of  the  Pelican 
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for  this  experiment  allowed  only  the  pilot  to  be  present  in  the  aircraft  during  the  flight, 
which  eliminated  the  chance  to  fly  and  test  the  interface.  A  secondary  plan  was  to  use  the 
interface  in  a  truck  during  the  TNT  experiment,  but  that  did  not  go  as  planned  either, 
since  it  was  determined  that  that  would  not  add  any  value  to  the  testing  process.  The  last 
plan  was  to  use  the  interface  during  the  final  day  of  the  TNT  experiment,  but  this  didn’t 
go  as  planned  either,  since  control  of  the  UAV  was  to  be  executed  by  the  team  in  the  van 
using  a  Unix-based  interface  and  all  that  could  be  done  in  the  Tactical  Operational  Center 
(TOC)  was  to  observe  the  process  and  message  using  the  current  interface. 

All  those  factors  eliminate  the  chance  to  evaluate  the  SA  view  changes.  But  at 
least  one  of  the  proposed  changes  was  implemented  and  used.  Eugene  Bourakov  changed 
the  Message  Box  to  flash  when  a  message  is  received.  Even  though  a  formal  evaluation 
process  was  not  executed  for  this  added  feature,  the  initial  user  response  was  positive.  In 
addition,  a  finger  roll-type  mouse  is  easy  to  implement  and  it  is  expected  that  it  can  be 
used  in  similar  conditions,  regardless  of  the  implementation. 

The  other  proposed  additions  should  create  a  SA  increase  with  relatively  low  cost. 
However  further  testing  is  necessary  in  order  to  test  these  ideas.  The  features  proposed 
here  can  be  used  as  references  for  future  SA  view  designs  for  the  NPS  Rascal  UAV,  and 
will  hopefully  make  the  operators’  job  easier  by  anning  him/her  with  better  SA. 
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VI.  CONCLUSION  SUGGESTIONS 


This  research  was  undertaken  with  the  goal  of  improving  the  SA  of  the  UAV 
operators,  in  this  particular  case,  the  operators  of  the  Rascal  UAV  used  by  the  NPS  TNT 
project.  By  conducting  a  literature  review  of  SA  and  interfaces,  followed  by  the  history  of 
UAV  and  control  over  the  network,  the  shortcomings  of  the  SA  view  currently  used  for 
the  Rascal  UAV  were  identified  in  order  to  provide  the  operator  with  good  SA  and  ease 
of  use  for  operational  missions. 

The  specific  problem  areas  identified  during  the  research  in  this  phase  were 
classified  our  findings  into  three  categories: 

•  Problems  regarding  the  SA  of  the  operator; 

•  Problems  due  to  nature  of  flight  conditions  ; 

•  Problems  due  to  lack  of  alerts  to  the  operator. 

Classifying  the  problem  areas  was  followed  by  developing  candidate  solutions  for 
those  problematic  areas.  The  proposed  solutions  aimed  to  be  easy  and  relatively  cheap, 
both  in  terms  of  money  and  in  terms  of  time,  to  implement.  The  proposed  solutions  are: 

•  Use  of  an  FOV  frame  on  the  map; 

•  Use  of  a  north  arrow; 

•  Use  of  a  grid  over  the  map; 

•  Use  of  a  hood  for  changing  light  conditions; 

•  Use  of  a  finger  roll  mouse; 

•  Use  of  a  flashing  message  box  for  incoming  messages; 

•  Use  of  a  “Master  Caution”  light  and  panel; 

Unfortunately,  there  was  no  chance  to  test  these  proposed  solutions  or  see  all  of 
them  implemented;  however,  they  should  provide  guidelines  for  addressing  deficiencies 
in  current  SA  views  and  developing  pilot  SA  interface  for  future  analyze. 
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